-- Logs begin at Mon 2022-06-27 15:05:11 UTC, end at Mon 2022-06-27 16:10:11 UTC. -- Jun 27 15:33:11.626492 dv-cinder-847801-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 27 15:33:11.636401 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 27 15:33:11.638460 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jun 27 15:33:11 2022] *** Jun 27 15:33:11.638460 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 27 15:33:11.638460 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 27 15:33:11.638460 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: nodename: dv-cinder-847801-1-iscsi Jun 27 15:33:11.638460 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: machine: x86_64 Jun 27 15:33:11.638460 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: clock source: unix Jun 27 15:33:11.638460 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: pcre jit disabled Jun 27 15:33:11.638460 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: detected number of CPU cores: 4 Jun 27 15:33:11.638460 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: current working directory: / Jun 27 15:33:11.638460 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: detected binary path: /usr/bin/uwsgi-core Jun 27 15:33:11.638999 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: your processes number limit is 47680 Jun 27 15:33:11.638999 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: your memory page size is 4096 bytes Jun 27 15:33:11.638999 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: detected max file descriptor number: 2048 Jun 27 15:33:11.638999 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: lock engine: pthread robust mutexes Jun 27 15:33:11.638999 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: thunder lock: enabled Jun 27 15:33:11.638999 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 27 15:33:11.638999 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 27 15:33:11.683229 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: Python main interpreter initialized at 0x55f864de19c0 Jun 27 15:33:11.683229 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: python threads support enabled Jun 27 15:33:11.683229 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: your server socket listen backlog is limited to 100 connections Jun 27 15:33:11.683229 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: your mercy for graceful operations on workers is 90 seconds Jun 27 15:33:11.683574 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: mapped 403077 bytes (393 KB) for 2 cores Jun 27 15:33:11.683652 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: *** Operational MODE: preforking *** Jun 27 15:33:11.683930 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: *** uWSGI is running in multiple interpreter mode *** Jun 27 15:33:11.683930 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: spawned uWSGI master process (pid: 91934) Jun 27 15:33:11.684129 dv-cinder-847801-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 27 15:33:11.687949 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: spawned uWSGI worker 1 (pid: 91941, cores: 1) Jun 27 15:33:11.687949 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: spawned uWSGI worker 2 (pid: 91942, cores: 1) Jun 27 15:33:11.687949 dv-cinder-847801-1-iscsi devstack@keystone.service[91934]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 27 15:33:17.177224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 15:33:17.177224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: from cryptography.utils import int_from_bytes Jun 27 15:33:17.178053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 15:33:17.178053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: from cryptography.utils import int_from_bytes Jun 27 15:33:17.178465 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 15:33:17.178465 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: from cryptography.utils import int_from_bytes Jun 27 15:33:17.178565 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 15:33:17.178565 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: from cryptography.utils import int_from_bytes Jun 27 15:33:17.228908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 27 15:33:17.229116 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 27 15:33:17.229268 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] command line args: [] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 27 15:33:17.229469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 27 15:33:17.229560 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 27 15:33:17.229689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 27 15:33:17.229783 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ================================================================================ {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 27 15:33:17.229890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] command line args: [] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 27 15:33:17.229970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] admin_token = **** {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.230162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 27 15:33:17.230540 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] config_dir = [] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.230623 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ================================================================================ {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 27 15:33:17.230696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.230696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] config_source = [] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.230826 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] admin_token = **** {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.230902 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] debug = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.230983 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] config_dir = [] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.231057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.231274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.231274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] config_source = [] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.231409 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.231491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] debug = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.231551 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.231607 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.231687 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.231751 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.231921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.232027 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.232123 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] list_limit = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.232229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.232325 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] log_config_append = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.232417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.232518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.232620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] list_limit = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.232716 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] log_dir = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.232809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] log_config_append = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.232908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] log_file = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.233013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.233129 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.233210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] log_dir = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.233333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.233414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] log_file = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.233534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.233609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.233771 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.233850 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.233959 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.234043 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.234156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.234278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.234364 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.234446 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.234518 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.234592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.234721 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.234809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.234952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.235031 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.235181 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.235265 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.235411 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.235507 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.235638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.235714 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.235868 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.236031 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.236168 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.236257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.236335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.236597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.236678 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.236794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] publish_errors = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.237015 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.237106 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.237245 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.237334 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] publish_errors = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.237412 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.237483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.237673 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.237755 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.237882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.238020 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.238143 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.238264 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] standard_threads = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.238379 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.238551 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.238669 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] standard_threads = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.238782 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.238782 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.239005 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.239118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] use_journal = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.239244 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.239360 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] use_json = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.239479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.239598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] use_stderr = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.239713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] use_journal = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.239832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] use_syslog = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.239958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] use_json = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.240074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.240193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] use_stderr = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.240193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] use_syslog = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.240419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.240534 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 15:33:17.240647 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.240761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.240880 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.240994 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.241108 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.241227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.241348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.241469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.241589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.241792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.241892 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.241987 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.242090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.242189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.242345 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.242993 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.243203 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.243385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.243549 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.243683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.243810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.243928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.243928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.244144 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.244413 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.244666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.244790 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.244906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.245021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.245141 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.245331 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.245451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.245566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.245722 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.245839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.245966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.246091 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.246207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.246328 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.246443 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.246558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.246672 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.246797 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.246918 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.247034 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.247149 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.247262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.247384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.247508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.247625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.247740 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.247854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.247968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.248091 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.248213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.248329 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.248444 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.248559 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.248679 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.248803 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.248918 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.249039 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.249160 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.249281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.249396 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.249523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.249655 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.249773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.249887 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.250002 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.250126 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.250251 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.250373 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.250488 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.250603 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.250717 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.250831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.251014 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.251131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.251252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.251367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.251487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.251603 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.251726 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.251849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.251965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.252079 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.252193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.252306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.252431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.252547 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.252662 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.252760 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.252856 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.252951 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.253059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.253157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.253259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.253356 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.253452 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.253555 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.253730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.253836 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.253933 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.254032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.254110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.254263 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.254395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.254512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.254618 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.254733 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.254848 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.254952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.255073 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.255189 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.255294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.255407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.255528 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.255642 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.255758 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.255881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.255998 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.256121 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.256235 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.256351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.256465 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.256589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.256706 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.256830 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.256947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.257068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.257185 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.257299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.257422 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.257539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.257668 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.257785 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.257910 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.258030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.258147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.258262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.258377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.258498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.258615 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.258615 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.258846 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.258846 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.259053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.259167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.259282 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.259405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.259527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.259642 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.259757 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.259872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.259994 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.260118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.260235 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.260350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.260465 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.260580 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.260695 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.260818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.260935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.261041 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.261160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.261276 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.261391 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.261514 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.261630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.261804 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.261922 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.262038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.262154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.262340 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.262460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.262575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.262690 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.262805 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.262920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.263043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.263169 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.263298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.263415 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.263533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.263650 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.263765 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.263880 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.264006 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.264124 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.264241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.264356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.264471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.264586 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.264709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.264826 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.264941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.265056 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.265171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.265295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.265412 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.265533 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.265664 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.265782 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.265897 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.266024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] auth.external = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.266142 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.266259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.266374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.266489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.266611 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] auth.external = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.266736 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.266879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.267009 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] auth.password = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.267127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.267256 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] auth.token = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.267375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.267491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.267607 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] auth.password = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.267731 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.267848 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] auth.token = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.267963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.268078 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.268193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.268308 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.268435 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.268553 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.268669 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.268784 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.268907 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.269015 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.269141 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] credential.caching = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.269259 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.269374 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.269490 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.269612 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.269744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] credential.caching = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.269873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.269996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.270113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.270229 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.270344 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.270461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.270585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.270702 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.270818 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.270933 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.271049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.271165 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.271288 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.271405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.271522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.271637 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.271753 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.271868 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.271996 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.272113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.272229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.272344 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.272465 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.272582 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.272714 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.272832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.272947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] federation.caching = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.273062 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.273178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.273294 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.273421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.273559 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] federation.caching = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.273765 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.273957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.274169 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.274360 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.274559 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.274559 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.274793 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.274793 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.275013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.275130 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.275256 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.275374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.275491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.275608 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.275726 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.275844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.275970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.276088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.276205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] identity.caching = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.276322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.276438 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.276634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.276757 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.276875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] identity.caching = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.276998 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.277117 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.277234 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.277359 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.277477 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.277594 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.277729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.277847 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.277963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.278092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.278201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.278316 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.278433 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.278548 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.278663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.278788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.278916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.279034 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.279150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.279271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.279388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.279511 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.279629 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.279745 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.279860 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.279976 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.280092 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.280217 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.280335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.280451 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.280567 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.280683 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.280798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.280926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.281045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.281177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.281396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.281517 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.281679 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.281813 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.281930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.282047 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.282162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.282276 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.282390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.282514 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.282630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.282744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.282858 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.282972 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.283088 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.283210 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.283326 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.283441 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.283560 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.283676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.283791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.283914 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.284030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.284145 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.284259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.284374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.284749 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.284879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.284996 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.285111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.285227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.285347 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.285463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.285586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.285718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.285834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.285948 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.286062 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.286177 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.286304 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.286421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.286552 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.286715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.286834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.286951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.287076 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.287194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.287310 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.287427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.287542 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.287659 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.287783 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.287901 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.288016 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.288132 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.288247 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.288363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.288487 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.288606 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.288722 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.288836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.288952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.289077 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.289806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.289926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.290043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.290159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.290275 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.290406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.290525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.290642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.290758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.290874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.290990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.291115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.291233 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.291349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.291466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.291583 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.291700 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.291825 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.291943 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.292059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.292175 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.292291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.292407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.292533 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.292650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.292767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.292882 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.292999 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.293114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.293239 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.293357 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.293480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.293602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.293736 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.293853 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.293986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.294106 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.294222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.294339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.294455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.294572 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.294697 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.294815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.294932 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.295048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.295164 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.295280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.295405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.295572 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.295693 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.295810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.295927 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.296044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.296169 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.296287 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.296403 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.296519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.296635 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.296751 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.296877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.296995 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.297111 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.297227 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.297344 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.297461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.297585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.297735 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.297853 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.297970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.298086 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.298202 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.298330 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] resource.caching = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.298448 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.298565 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.298681 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.298798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.298914 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] resource.caching = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.299039 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.299157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.299274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.299391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.299508 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.299624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.299749 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.299868 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.299985 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.300101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.300217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.300332 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.300520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.300640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.300757 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] role.caching = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.300875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.300992 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] role.driver = sql {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.301109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.301234 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.301352 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] role.caching = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.301469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.301586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] role.driver = sql {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.301717 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.301836 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.301967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.302086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.302203 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.302320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.302437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.302562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.302680 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.302796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.302912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.303030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.303147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.303271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.303390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.303506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.303626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.303742 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.303858 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.303984 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.304104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.304221 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.304338 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.304454 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.304454 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.304683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.304890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.305528 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.305528 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.305866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.306082 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.306217 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.306338 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.306455 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.306579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.306697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.306814 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.306939 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.307057 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.307175 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.307292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.307408 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.307525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.307651 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.307770 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.307886 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.308004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.308004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.308233 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.308351 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.308468 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.308584 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.308700 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.308700 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.308928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.308928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.309148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.309266 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.309383 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.309499 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.309623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.309762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] token.caching = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.309878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.309995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.310110 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.310226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.310356 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.310476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.310592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] token.caching = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.310709 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.310826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.310942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.311068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.311186 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.311303 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.311420 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.311537 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.311653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.311778 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.311897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.312014 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.312130 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.312246 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.312363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.312489 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.312607 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.312723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.312839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.312957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.313074 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.313199 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.313317 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.313436 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.313552 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.313735 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.313735 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.313970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.314089 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.314205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.connection = **** {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.314330 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.314457 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.314457 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.314679 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.314795 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.314921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.connection = **** {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.315049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.315175 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.315294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.315411 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.315528 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.315645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.315770 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.315888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.316005 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.316122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.316239 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.316355 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.316481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.316600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.316717 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.316834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.316951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.317067 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.317193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.317311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.317429 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.317545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.317681 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.317801 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.317930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.318050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.318167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.318284 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.318401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.318525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.318653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91942) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 27 15:33:17.318773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.318773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.318773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 15:33:17.318773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91941) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 27 15:33:17.345876 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.346105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.346326 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.346411 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.346558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.346790 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.355740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.355874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.384872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.385102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.386332 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.386564 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.529706 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.531525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.579455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.579687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.580296 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.580529 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.643836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.644063 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.644286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.644388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.644505 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.644588 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.644719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.644799 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.644925 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.645018 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.645186 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.645404 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.670329 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.670583 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.670812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.670900 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.671035 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.671117 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.671255 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.671348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.671469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.671553 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.671682 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.671764 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.671893 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.672108 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.683643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.683751 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.697492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.697624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.697805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.697887 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.697962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.698035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.698162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.698254 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.698375 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.698458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.698587 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.698669 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.698796 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91942) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.698886 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91941) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 15:33:17.716720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.716893 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.716979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 15:33:17.717102 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 15:33:17.717254 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.717434 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.717517 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 15:33:17.717654 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 15:33:17.717739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.717928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.718013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.718187 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.718269 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.718442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.718523 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.718693 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.729324 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.729597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.729743 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.729889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.729987 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.730172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.730256 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.730445 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.738419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.738657 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.741724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.741958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.742042 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.742733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.742823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.743024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.743108 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.743299 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.743383 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.743571 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.743684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.743825 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.743911 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.744088 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.744169 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.744349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.744439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.744610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.744742 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.744871 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.744952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.745129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.771841 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.772564 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.773449 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.773788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.774265 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.774538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.779050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.779438 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.779763 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.779850 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.780231 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.780519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.787276 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.787554 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.788131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.788409 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.792686 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.792972 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.793252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.793532 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.793744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.793844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.794029 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.794110 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.794304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.794386 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.794580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.794662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.794848 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.795124 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.795392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.795654 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.814149 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.814427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.814921 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.815206 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.819197 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.819805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.823957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.824638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.824730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.824810 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.828803 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.829168 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.832429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.832795 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.836150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.836445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.836727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.836809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.837032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.837127 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.837270 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.837354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.837543 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.837627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.837923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.838215 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.857043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.857332 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.857596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.857760 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.857887 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.858042 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.858304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.858572 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.865735 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.866417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.868082 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.868357 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.868759 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.869037 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.871645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.871923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.872198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.872280 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.872474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.872566 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.872771 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.872856 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.873125 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.873402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.886391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.886655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.886755 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.886930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.887015 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.887210 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.887292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.887483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.888885 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.889156 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.889249 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.889443 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.889526 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.889746 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.907608 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.907828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.911376 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.911507 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.915434 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.915580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.917221 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.917316 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.918928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.919023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.919205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.919290 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.919484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.919576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.926801 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.926947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.928408 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.928515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.932251 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.932344 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.932542 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.932626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.932923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.933014 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.933197 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.933281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.942258 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.942343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.942545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.942630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.942821 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.942902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.951292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91942) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.951409 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91941) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 15:33:17.951566 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.951652 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.951841 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.951924 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.952110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.952205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.952377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.952468 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.952754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.952838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.953030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.953114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.953296 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.953388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.953562 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.953664 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.953863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.953945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.954130 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.954223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.954391 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.954474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.954651 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.954732 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.954904 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91942) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.954994 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91941) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 15:33:17.994313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f864de19c0 pid: 91941 (default app) Jun 27 15:33:17.995003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f864de19c0 pid: 91942 (default app) Jun 27 15:33:17.997788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f433b3-03a0-4745-9e43-cb4f1a8e73bf None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:17.998079 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f433b3-03a0-4745-9e43-cb4f1a8e73bf None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:17.998356 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f433b3-03a0-4745-9e43-cb4f1a8e73bf None None] PATH_INFO: `/v3` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:18.002530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Mon Jun 27 15:33:17 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:25.822490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bac6b1-edc5-49f0-846a-2c7df1eaca55 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:25.823053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bac6b1-edc5-49f0-846a-2c7df1eaca55 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:25.823053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bac6b1-edc5-49f0-846a-2c7df1eaca55 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:25.827357 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:25.832963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52886331-51e3-4dc9-8fac-94208204594f None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:25.833397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52886331-51e3-4dc9-8fac-94208204594f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:25.833512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52886331-51e3-4dc9-8fac-94208204594f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:25.902984 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: INFO dbcounter [None req-52886331-51e3-4dc9-8fac-94208204594f None None] Registered counter for database keystone Jun 27 15:33:25.924846 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG oslo_db.sqlalchemy.engines [None req-52886331-51e3-4dc9-8fac-94208204594f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91941) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 27 15:33:25.931220 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writer thread running {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 27 15:33:26.111319 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG passlib.handlers.bcrypt [None req-52886331-51e3-4dc9-8fac-94208204594f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91941) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 27 15:33:26.111741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG passlib.handlers.bcrypt [None req-52886331-51e3-4dc9-8fac-94208204594f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91941) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 27 15:33:26.168253 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-52886331-51e3-4dc9-8fac-94208204594f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:26.169894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-52886331-51e3-4dc9-8fac-94208204594f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:26.292721 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 27 15:33:26.455861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bac6b1-edc5-49f0-846a-2c7df1eaca55 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:26.526873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: INFO dbcounter [None req-21bac6b1-edc5-49f0-846a-2c7df1eaca55 None None] Registered counter for database keystone Jun 27 15:33:26.547190 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG oslo_db.sqlalchemy.engines [None req-21bac6b1-edc5-49f0-846a-2c7df1eaca55 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91942) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 27 15:33:26.553338 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writer thread running {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 27 15:33:26.675175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bac6b1-edc5-49f0-846a-2c7df1eaca55 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:26.679475 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85088c07-07e9-42b0-a07c-a547c42448fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:26.680213 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85088c07-07e9-42b0-a07c-a547c42448fb admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:26.680455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85088c07-07e9-42b0-a07c-a547c42448fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:26.680734 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85088c07-07e9-42b0-a07c-a547c42448fb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:26.689050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85088c07-07e9-42b0-a07c-a547c42448fb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:26.741573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: WARNING keystone.common.rbac_enforcer.enforcer [None req-85088c07-07e9-42b0-a07c-a547c42448fb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 27 15:33:26.741875 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85088c07-07e9-42b0-a07c-a547c42448fb admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:26.752382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Mon Jun 27 15:33:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 15:33:26.758437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52886331-51e3-4dc9-8fac-94208204594f None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:26.772985 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52886331-51e3-4dc9-8fac-94208204594f None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:26.778064 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d70b78-878a-4669-b202-cc7193fdfe2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:26.779184 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d70b78-878a-4669-b202-cc7193fdfe2c admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:26.779463 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d70b78-878a-4669-b202-cc7193fdfe2c admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:26.779736 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d70b78-878a-4669-b202-cc7193fdfe2c admin admin] PATH_INFO: `/v3/projects/53048925682f4806b3831f88058d3439` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:26.783826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d70b78-878a-4669-b202-cc7193fdfe2c admin admin] RBAC: Authorizing `identity:get_project(project_id=53048925682f4806b3831f88058d3439)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:26.840717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d8d70b78-878a-4669-b202-cc7193fdfe2c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 27 15:33:26.841195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d70b78-878a-4669-b202-cc7193fdfe2c admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:26.845836 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Mon Jun 27 15:33:26 2022] GET /identity/v3/projects/53048925682f4806b3831f88058d3439 => generated 344 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:28.884775 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010efc12-5a5a-4090-b8d0-fa7523eb1e05 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:28.885288 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010efc12-5a5a-4090-b8d0-fa7523eb1e05 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:28.885377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010efc12-5a5a-4090-b8d0-fa7523eb1e05 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:28.889468 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:28.894943 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230a3ab-21df-4be4-804f-16984ab9d36b None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:28.895383 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230a3ab-21df-4be4-804f-16984ab9d36b None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:28.895501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230a3ab-21df-4be4-804f-16984ab9d36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:28.915348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-9230a3ab-21df-4be4-804f-16984ab9d36b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:28.916889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-9230a3ab-21df-4be4-804f-16984ab9d36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:28.924669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 27 15:33:28.946783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010efc12-5a5a-4090-b8d0-fa7523eb1e05 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:28.958749 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010efc12-5a5a-4090-b8d0-fa7523eb1e05 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:28.963450 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69053fc0-d2ae-4442-8e86-348c35612967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:28.964084 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69053fc0-d2ae-4442-8e86-348c35612967 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:28.964369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69053fc0-d2ae-4442-8e86-348c35612967 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:28.964650 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69053fc0-d2ae-4442-8e86-348c35612967 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:28.972386 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69053fc0-d2ae-4442-8e86-348c35612967 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:28.974227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69053fc0-d2ae-4442-8e86-348c35612967 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:28.983663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Mon Jun 27 15:33:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 15:33:28.989340 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230a3ab-21df-4be4-804f-16984ab9d36b None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:28.998619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230a3ab-21df-4be4-804f-16984ab9d36b None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:29.002939 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2329738-857d-4057-ae70-e92bdde0dbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:29.003803 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2329738-857d-4057-ae70-e92bdde0dbb5 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:29.004090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2329738-857d-4057-ae70-e92bdde0dbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:29.004372 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2329738-857d-4057-ae70-e92bdde0dbb5 admin admin] PATH_INFO: `/v3/users/4b433ad0233c4e5dbe8fad9896962e25` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:29.005790 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2329738-857d-4057-ae70-e92bdde0dbb5 admin admin] RBAC: Authorizing `identity:get_user(user_id=4b433ad0233c4e5dbe8fad9896962e25)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:29.007605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2329738-857d-4057-ae70-e92bdde0dbb5 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:29.009166 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Mon Jun 27 15:33:28 2022] GET /identity/v3/users/4b433ad0233c4e5dbe8fad9896962e25 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:31.146839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44c456-fa6c-47a2-85c4-65e7f2115e70 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:31.147376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44c456-fa6c-47a2-85c4-65e7f2115e70 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:31.147376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44c456-fa6c-47a2-85c4-65e7f2115e70 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:31.148755 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:31.150090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87964820-90c6-436a-90d5-9df625b9b034 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:31.150486 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87964820-90c6-436a-90d5-9df625b9b034 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:31.150715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87964820-90c6-436a-90d5-9df625b9b034 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:31.152077 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:33:31.154211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c9d79-3241-43a7-872b-e299b594687a None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:31.154503 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c9d79-3241-43a7-872b-e299b594687a None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:31.154786 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c9d79-3241-43a7-872b-e299b594687a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:31.157126 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873adb6-f3d8-485c-b829-331dcac2c231 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:31.157421 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873adb6-f3d8-485c-b829-331dcac2c231 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:31.157738 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873adb6-f3d8-485c-b829-331dcac2c231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:31.175002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-8873adb6-f3d8-485c-b829-331dcac2c231 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:31.176183 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-8873adb6-f3d8-485c-b829-331dcac2c231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:31.194861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG passlib.handlers.bcrypt [None req-059c9d79-3241-43a7-872b-e299b594687a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91942) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 27 15:33:31.195269 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG passlib.handlers.bcrypt [None req-059c9d79-3241-43a7-872b-e299b594687a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91942) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 27 15:33:31.241416 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-059c9d79-3241-43a7-872b-e299b594687a None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:31.243265 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-059c9d79-3241-43a7-872b-e299b594687a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:31.257482 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 7/12] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:33:31.281912 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873adb6-f3d8-485c-b829-331dcac2c231 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:31.305659 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:31.344960 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 6/13] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:33:31.369542 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c9d79-3241-43a7-872b-e299b594687a None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:31.388144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873adb6-f3d8-485c-b829-331dcac2c231 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:31.422882 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf330413-62a3-40ab-95bc-d5fdd09d3859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:31.423604 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf330413-62a3-40ab-95bc-d5fdd09d3859 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:31.423891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf330413-62a3-40ab-95bc-d5fdd09d3859 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:31.424170 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf330413-62a3-40ab-95bc-d5fdd09d3859 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:31.436003 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf330413-62a3-40ab-95bc-d5fdd09d3859 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:33:31.447317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-bf330413-62a3-40ab-95bc-d5fdd09d3859 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:33:31.447317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:31.447317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:31.447317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:31.447317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:31.447317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:31.447317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:31.447317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:31.447317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:31.447317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:31.447317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:31.447317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:31.447317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:33:31.447317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:33:31.447317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:33:31.447317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:33:31.448273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:33:31.448273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:33:31.448273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:33:31.448273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:33:31.448273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:31.448273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:31.448273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:31.448273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:31.448273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:31.448273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:31.448273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:31.448273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:31.448273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:31.448273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:31.448273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:31.448273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:31.449059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:31.449059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:31.449059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:31.449059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:31.449059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:33:31.449059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:33:31.449059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:33:31.449059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:33:31.449059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:33:31.449059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:33:31.449059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:33:31.449059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:33:31.449059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 8/14] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:33:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:33:31.454341 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf330413-62a3-40ab-95bc-d5fdd09d3859 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:31.472254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c9d79-3241-43a7-872b-e299b594687a None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:31.476740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:31.516360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f3cf3-532e-4a09-8669-f35235017c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:31.517261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f3cf3-532e-4a09-8669-f35235017c28 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:31.517556 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f3cf3-532e-4a09-8669-f35235017c28 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:31.517866 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f3cf3-532e-4a09-8669-f35235017c28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:31.546236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:31.565605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf330413-62a3-40ab-95bc-d5fdd09d3859 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:31.587957 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f3cf3-532e-4a09-8669-f35235017c28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:31.589494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f3cf3-532e-4a09-8669-f35235017c28 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:31.599506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1142b-7678-4b2f-ab85-1a71c7a8ce06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:31.600194 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1142b-7678-4b2f-ab85-1a71c7a8ce06 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:31.600487 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1142b-7678-4b2f-ab85-1a71c7a8ce06 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:31.600770 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1142b-7678-4b2f-ab85-1a71c7a8ce06 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:31.601533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1142b-7678-4b2f-ab85-1a71c7a8ce06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:31.601879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1142b-7678-4b2f-ab85-1a71c7a8ce06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:31.603851 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a1142b-7678-4b2f-ab85-1a71c7a8ce06 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:31.615989 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 9/15] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:33:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:33:31.622219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1142b-7678-4b2f-ab85-1a71c7a8ce06 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:31.690119 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 7/16] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:33:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 15:33:31.696158 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f3cf3-532e-4a09-8669-f35235017c28 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:31.699714 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:31.733068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1142b-7678-4b2f-ab85-1a71c7a8ce06 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:31.769019 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4699fd-13c0-4d45-8f1f-bed8e4aea49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:31.769925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4699fd-13c0-4d45-8f1f-bed8e4aea49d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:31.770213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4699fd-13c0-4d45-8f1f-bed8e4aea49d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:31.770490 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4699fd-13c0-4d45-8f1f-bed8e4aea49d None admin] PATH_INFO: `/v3/users/4b433ad0233c4e5dbe8fad9896962e25` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:31.771697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4699fd-13c0-4d45-8f1f-bed8e4aea49d None admin] RBAC: Authorizing `identity:get_user(user_id=4b433ad0233c4e5dbe8fad9896962e25)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:31.773153 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4699fd-13c0-4d45-8f1f-bed8e4aea49d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:31.775310 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 10/17] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:33:31 2022] GET /identity/v3/users/4b433ad0233c4e5dbe8fad9896962e25 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:33:31.780800 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4699fd-13c0-4d45-8f1f-bed8e4aea49d None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:31.795410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:31.805898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f3cf3-532e-4a09-8669-f35235017c28 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:31.842716 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e0c4b-297a-47a2-9a16-a9fd67dd13a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:31.843519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e0c4b-297a-47a2-9a16-a9fd67dd13a7 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:31.843805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e0c4b-297a-47a2-9a16-a9fd67dd13a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:31.844083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e0c4b-297a-47a2-9a16-a9fd67dd13a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:31.862818 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e0c4b-297a-47a2-9a16-a9fd67dd13a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:31.864299 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e0c4b-297a-47a2-9a16-a9fd67dd13a7 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:31.885060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4699fd-13c0-4d45-8f1f-bed8e4aea49d None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:31.886539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-b66e0c4b-297a-47a2-9a16-a9fd67dd13a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:33:31.886539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:31.886539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:31.886539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:31.886539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:31.886539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:31.886539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:31.886539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:31.886539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:31.886539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:31.886539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:31.886539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:31.886539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:33:31.886539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:33:31.886539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:33:31.887484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:33:31.887484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:31.887484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:31.887484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:31.887484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:31.887484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:31.887484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:31.887484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:31.887484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:31.887484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:31.887484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:31.887484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:31.887484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:31.887484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:31.887484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:31.887484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:31.887484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:31.888515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:33:31.888515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:33:31.888515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:33:31.888515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:33:31.888515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 8/18] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:33:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:33:31.893919 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e0c4b-297a-47a2-9a16-a9fd67dd13a7 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:31.926909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ec7be-e593-404f-b0b0-687a0f46c7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:31.927734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ec7be-e593-404f-b0b0-687a0f46c7e7 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:31.928021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ec7be-e593-404f-b0b0-687a0f46c7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:31.928299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ec7be-e593-404f-b0b0-687a0f46c7e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:31.929566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ec7be-e593-404f-b0b0-687a0f46c7e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:31.932286 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ec7be-e593-404f-b0b0-687a0f46c7e7 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:31.934198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 11/19] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:33:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:33:31.940223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ec7be-e593-404f-b0b0-687a0f46c7e7 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:31.956098 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:32.004292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e0c4b-297a-47a2-9a16-a9fd67dd13a7 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:32.029948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:32.044289 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bca26-bf1d-448a-9bbd-75ab427df256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:32.044971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bca26-bf1d-448a-9bbd-75ab427df256 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:32.045258 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bca26-bf1d-448a-9bbd-75ab427df256 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:32.045537 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bca26-bf1d-448a-9bbd-75ab427df256 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:32.046314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bca26-bf1d-448a-9bbd-75ab427df256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:32.046627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bca26-bf1d-448a-9bbd-75ab427df256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:32.048042 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bca26-bf1d-448a-9bbd-75ab427df256 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:32.054125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ec7be-e593-404f-b0b0-687a0f46c7e7 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:32.069074 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 9/20] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:33:32.092184 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3025d-9fff-4e91-b920-1331b82a7e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:32.092880 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3025d-9fff-4e91-b920-1331b82a7e7f None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:32.093172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3025d-9fff-4e91-b920-1331b82a7e7f None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:32.093457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3025d-9fff-4e91-b920-1331b82a7e7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:32.094362 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3025d-9fff-4e91-b920-1331b82a7e7f None admin] RBAC: Adding query filter params (role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.domain.id=default, user.id=4b433ad0233c4e5dbe8fad9896962e25) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:32.094683 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3025d-9fff-4e91-b920-1331b82a7e7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:32.096206 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3025d-9fff-4e91-b920-1331b82a7e7f None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:32.104395 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 12/21] 10.222.0.20 () {56 vars in 1322 bytes} [Mon Jun 27 15:33:31 2022] GET /identity/v3/role_assignments?user.id=4b433ad0233c4e5dbe8fad9896962e25&scope.domain.id=default&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 233 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:33:34.481491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23783a-911e-4710-9f90-ce562bad124b None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:34.482525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23783a-911e-4710-9f90-ce562bad124b None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:34.483117 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23783a-911e-4710-9f90-ce562bad124b None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:34.484720 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 10/22] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:34.490350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1545c-c3c5-40cf-a90c-8ee9e5749e8b None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:34.490838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1545c-c3c5-40cf-a90c-8ee9e5749e8b None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:34.490961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1545c-c3c5-40cf-a90c-8ee9e5749e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:34.514710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-25b1545c-c3c5-40cf-a90c-8ee9e5749e8b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:34.516260 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-25b1545c-c3c5-40cf-a90c-8ee9e5749e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:34.619178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 13/23] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:34.658748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23783a-911e-4710-9f90-ce562bad124b None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:34.810006 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23783a-911e-4710-9f90-ce562bad124b None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:34.840088 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:34.866404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd576df6-227f-4b70-9c95-361ab77f12a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:34.867221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd576df6-227f-4b70-9c95-361ab77f12a7 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:34.867506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd576df6-227f-4b70-9c95-361ab77f12a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:34.868358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd576df6-227f-4b70-9c95-361ab77f12a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:34.884372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd576df6-227f-4b70-9c95-361ab77f12a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:34.885886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd576df6-227f-4b70-9c95-361ab77f12a7 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:34.893122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87779068-3087-4975-bd06-226eeeb3ccd3 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:34.893420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87779068-3087-4975-bd06-226eeeb3ccd3 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:34.893732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87779068-3087-4975-bd06-226eeeb3ccd3 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:34.895123 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 14/24] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:34.907139 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cd428-3212-4088-ad33-abe2f4b05cae None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:34.907711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cd428-3212-4088-ad33-abe2f4b05cae None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:34.908186 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cd428-3212-4088-ad33-abe2f4b05cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:34.911655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-cd576df6-227f-4b70-9c95-361ab77f12a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:33:34.911655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:34.911655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:34.911655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:34.911655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:34.911655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:34.911655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:34.911655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:34.911655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:34.911655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:34.911655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:34.911655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:34.911655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:33:34.911655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:33:34.911655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:33:34.912431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:33:34.912431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:34.912431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:34.912431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:34.912431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:34.912431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:34.912431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:34.912431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:34.912431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:34.912431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:34.912431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:34.912431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:34.912431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:34.912431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:34.912431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:34.912431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:34.912431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:34.913216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:33:34.913216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:33:34.913216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:33:34.913216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:33:34.913216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 11/25] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:33:34.918971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd576df6-227f-4b70-9c95-361ab77f12a7 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:34.947106 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-f89cd428-3212-4088-ad33-abe2f4b05cae None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:34.948631 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f89cd428-3212-4088-ad33-abe2f4b05cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:35.068055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd576df6-227f-4b70-9c95-361ab77f12a7 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:35.151600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a20335-fffe-4323-9b4b-6e12f41dbf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:35.152323 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a20335-fffe-4323-9b4b-6e12f41dbf89 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:35.152644 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a20335-fffe-4323-9b4b-6e12f41dbf89 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:35.153028 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a20335-fffe-4323-9b4b-6e12f41dbf89 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:35.153883 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a20335-fffe-4323-9b4b-6e12f41dbf89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:35.154308 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a20335-fffe-4323-9b4b-6e12f41dbf89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:35.156601 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a20335-fffe-4323-9b4b-6e12f41dbf89 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:35.195579 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 15/26] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:35.199405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 12/27] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:35.205195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89cd428-3212-4088-ad33-abe2f4b05cae None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:35.213177 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf0a3b-cbea-4013-878e-ef598dd7e081 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:35.213470 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf0a3b-cbea-4013-878e-ef598dd7e081 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:35.213779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf0a3b-cbea-4013-878e-ef598dd7e081 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:35.213904 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:35.215153 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 13/28] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:33:35.220670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd69fd-4003-42f1-a469-3936401a3577 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:35.221154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd69fd-4003-42f1-a469-3936401a3577 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:35.221579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd69fd-4003-42f1-a469-3936401a3577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:35.250937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-8efd69fd-4003-42f1-a469-3936401a3577 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:35.252412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8efd69fd-4003-42f1-a469-3936401a3577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:35.280099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:35.344444 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 14/29] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 27 15:33:35.345328 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd69fd-4003-42f1-a469-3936401a3577 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:35.425182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89cd428-3212-4088-ad33-abe2f4b05cae None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:35.467503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc9563-52ab-4b33-ae08-e2e2440d5aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:35.468418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc9563-52ab-4b33-ae08-e2e2440d5aad None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:35.468854 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc9563-52ab-4b33-ae08-e2e2440d5aad None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:35.469277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc9563-52ab-4b33-ae08-e2e2440d5aad None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:35.470293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc9563-52ab-4b33-ae08-e2e2440d5aad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:35.472065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc9563-52ab-4b33-ae08-e2e2440d5aad None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:35.510683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd69fd-4003-42f1-a469-3936401a3577 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:35.551915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfa822-fd45-4135-89ec-17b3ae91f378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:35.553019 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfa822-fd45-4135-89ec-17b3ae91f378 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:35.553508 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfa822-fd45-4135-89ec-17b3ae91f378 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:35.553945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfa822-fd45-4135-89ec-17b3ae91f378 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:35.566816 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfa822-fd45-4135-89ec-17b3ae91f378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:33:35.576245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-28cfa822-fd45-4135-89ec-17b3ae91f378 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:33:35.576245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:35.576245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:35.576245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:35.576245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:35.576245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:35.576245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:35.576245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:35.576245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:35.576245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:35.576245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:35.576245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:35.576245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:33:35.576245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:33:35.576245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:33:35.576245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:33:35.577291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:33:35.577291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:33:35.577291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:33:35.577291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:33:35.577291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:35.577291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:35.577291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:35.577291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:35.577291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:35.577291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:35.577291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:35.577291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:35.577291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:35.577291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:35.577291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:35.577291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:35.578166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:35.578166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:35.578166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:35.578166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:35.578166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:33:35.578166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:33:35.578166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:33:35.578166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:33:35.578166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:33:35.578166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:33:35.578166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:33:35.578166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:33:35.580110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 15/30] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:33:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:33:35.581197 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfa822-fd45-4135-89ec-17b3ae91f378 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:35.603942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:35.633566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 16/31] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jun 27 15:33:35 2022] POST /identity/v3/projects => generated 301 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:33:35.633566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce7367-754b-42f9-9237-b4db29e3d26a None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:35.633566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce7367-754b-42f9-9237-b4db29e3d26a None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:35.633566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce7367-754b-42f9-9237-b4db29e3d26a None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:35.633566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 17/32] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:35.634668 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce7367-754b-42f9-9237-b4db29e3d26a None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:35.739554 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:35.773414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfa822-fd45-4135-89ec-17b3ae91f378 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:35.832806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce7367-754b-42f9-9237-b4db29e3d26a None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:35.849978 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec11ff-79bf-4acf-8dec-c7a4b663cd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:35.850668 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec11ff-79bf-4acf-8dec-c7a4b663cd23 None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:35.850952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec11ff-79bf-4acf-8dec-c7a4b663cd23 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:35.851234 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec11ff-79bf-4acf-8dec-c7a4b663cd23 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:35.852001 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec11ff-79bf-4acf-8dec-c7a4b663cd23 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:35.853471 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec11ff-79bf-4acf-8dec-c7a4b663cd23 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:35.888292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e2cf8-35c8-444e-9a55-9e2b986887d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:35.888899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e2cf8-35c8-444e-9a55-9e2b986887d7 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:35.889191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e2cf8-35c8-444e-9a55-9e2b986887d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:35.889473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e2cf8-35c8-444e-9a55-9e2b986887d7 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:35.890278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e2cf8-35c8-444e-9a55-9e2b986887d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:35.890606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e2cf8-35c8-444e-9a55-9e2b986887d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:35.892183 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e2cf8-35c8-444e-9a55-9e2b986887d7 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:35.904290 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 18/33] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:33:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:35.905546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1295d-fca0-41fe-a325-160afae9fdd3 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:35.906668 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1295d-fca0-41fe-a325-160afae9fdd3 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:35.907009 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1295d-fca0-41fe-a325-160afae9fdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:35.927216 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-4ea1295d-fca0-41fe-a325-160afae9fdd3 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:35.928769 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4ea1295d-fca0-41fe-a325-160afae9fdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:35.999695 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 16/34] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Jun 27 15:33:35 2022] POST /identity/v3/roles => generated 225 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:33:36.001139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec11ff-79bf-4acf-8dec-c7a4b663cd23 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:36.012112 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 19/35] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:36.018056 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f7655-b374-4c41-8ec6-92bc5198cc44 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:36.018056 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f7655-b374-4c41-8ec6-92bc5198cc44 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:36.018056 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f7655-b374-4c41-8ec6-92bc5198cc44 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:36.018056 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 20/36] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:36.025758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d5c40-b279-4ff4-8112-2b458e436f2f None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:36.026425 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d5c40-b279-4ff4-8112-2b458e436f2f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:36.026879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d5c40-b279-4ff4-8112-2b458e436f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:36.045573 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-924d5c40-b279-4ff4-8112-2b458e436f2f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:36.047008 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-924d5c40-b279-4ff4-8112-2b458e436f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:36.106843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:36.120148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:36.146082 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 21/37] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:33:36.146082 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d5c40-b279-4ff4-8112-2b458e436f2f None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:36.157192 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec11ff-79bf-4acf-8dec-c7a4b663cd23 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:36.198155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd940d-edd4-44d6-8038-e7d6e70b3b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:36.199046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd940d-edd4-44d6-8038-e7d6e70b3b02 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:36.199327 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd940d-edd4-44d6-8038-e7d6e70b3b02 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:36.199602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd940d-edd4-44d6-8038-e7d6e70b3b02 None admin] PATH_INFO: `/v3/users/4b433ad0233c4e5dbe8fad9896962e25` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:36.201293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd940d-edd4-44d6-8038-e7d6e70b3b02 None admin] RBAC: Authorizing `identity:get_user(user_id=4b433ad0233c4e5dbe8fad9896962e25)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:36.202934 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd940d-edd4-44d6-8038-e7d6e70b3b02 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:36.206576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 17/38] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:33:36 2022] GET /identity/v3/users/4b433ad0233c4e5dbe8fad9896962e25 => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:36.209209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef5997-44e3-41f4-a92d-af49e80e0212 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:36.211892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef5997-44e3-41f4-a92d-af49e80e0212 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:36.211892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef5997-44e3-41f4-a92d-af49e80e0212 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:36.213868 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 18/39] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:36.214608 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef5997-44e3-41f4-a92d-af49e80e0212 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:36.266268 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d5c40-b279-4ff4-8112-2b458e436f2f None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:36.300957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac275c9-a412-470a-87b6-85e3426264fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:36.301652 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac275c9-a412-470a-87b6-85e3426264fe None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:36.301949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac275c9-a412-470a-87b6-85e3426264fe None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:36.302230 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac275c9-a412-470a-87b6-85e3426264fe None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:36.302982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac275c9-a412-470a-87b6-85e3426264fe None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:36.304372 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac275c9-a412-470a-87b6-85e3426264fe None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:36.319827 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 22/40] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Jun 27 15:33:36 2022] POST /identity/v3/roles => generated 219 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:33:36.320693 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac275c9-a412-470a-87b6-85e3426264fe None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:36.342669 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef5997-44e3-41f4-a92d-af49e80e0212 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:36.375200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:36.389810 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2fc52-b532-4ce8-b856-5b9b148b179d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:36.390507 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2fc52-b532-4ce8-b856-5b9b148b179d None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:36.390796 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2fc52-b532-4ce8-b856-5b9b148b179d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:36.391180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2fc52-b532-4ce8-b856-5b9b148b179d None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:36.391867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2fc52-b532-4ce8-b856-5b9b148b179d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:36.393321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2fc52-b532-4ce8-b856-5b9b148b179d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:36.410011 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 19/41] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Jun 27 15:33:36 2022] POST /identity/v3/roles => generated 223 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:33:36.412038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5548d-9137-4036-b769-925628092f32 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:36.412520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5548d-9137-4036-b769-925628092f32 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:36.412990 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5548d-9137-4036-b769-925628092f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:36.434933 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:36.447189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-4ee5548d-9137-4036-b769-925628092f32 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:36.449491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-4ee5548d-9137-4036-b769-925628092f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:36.468150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac275c9-a412-470a-87b6-85e3426264fe None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:36.541357 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 20/42] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:36.568415 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5548d-9137-4036-b769-925628092f32 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:36.627227 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265eebd-6b5b-4fa9-8fb8-76ff9a0e34ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:36.628157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265eebd-6b5b-4fa9-8fb8-76ff9a0e34ee None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:36.628443 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265eebd-6b5b-4fa9-8fb8-76ff9a0e34ee None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:36.628834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265eebd-6b5b-4fa9-8fb8-76ff9a0e34ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:36.630092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265eebd-6b5b-4fa9-8fb8-76ff9a0e34ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:36.631610 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265eebd-6b5b-4fa9-8fb8-76ff9a0e34ee None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:36.633328 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 23/43] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:33:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:36.642101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265eebd-6b5b-4fa9-8fb8-76ff9a0e34ee None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:36.697332 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5548d-9137-4036-b769-925628092f32 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:36.736292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067b09b-9a2c-4387-8fb4-f1f571201360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:36.737100 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067b09b-9a2c-4387-8fb4-f1f571201360 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:36.737388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067b09b-9a2c-4387-8fb4-f1f571201360 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:36.737694 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067b09b-9a2c-4387-8fb4-f1f571201360 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:36.738890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067b09b-9a2c-4387-8fb4-f1f571201360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:36.740356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067b09b-9a2c-4387-8fb4-f1f571201360 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:36.742028 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 21/44] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:33:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:33:36.748160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067b09b-9a2c-4387-8fb4-f1f571201360 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:36.751775 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:36.769966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265eebd-6b5b-4fa9-8fb8-76ff9a0e34ee None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:36.777102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:36.806471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a244e24-6ba1-4ef1-a864-4cef91a36a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:36.807359 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a244e24-6ba1-4ef1-a864-4cef91a36a79 None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:36.807652 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a244e24-6ba1-4ef1-a864-4cef91a36a79 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:36.807948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a244e24-6ba1-4ef1-a864-4cef91a36a79 None admin] PATH_INFO: `/v3/domains/default/users/4b433ad0233c4e5dbe8fad9896962e25/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:36.810056 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a244e24-6ba1-4ef1-a864-4cef91a36a79 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4b433ad0233c4e5dbe8fad9896962e25, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:36.811519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a244e24-6ba1-4ef1-a864-4cef91a36a79 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:36.827651 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 24/45] 10.222.0.20 () {58 vars in 1361 bytes} [Mon Jun 27 15:33:36 2022] PUT /identity/v3/domains/default/users/4b433ad0233c4e5dbe8fad9896962e25/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:33:36.855289 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067b09b-9a2c-4387-8fb4-f1f571201360 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:36.894721 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03efeee-493a-4580-b34c-86296b575d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:36.895414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03efeee-493a-4580-b34c-86296b575d79 None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:36.895705 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03efeee-493a-4580-b34c-86296b575d79 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:36.895989 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03efeee-493a-4580-b34c-86296b575d79 None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:36.896744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03efeee-493a-4580-b34c-86296b575d79 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:36.898318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03efeee-493a-4580-b34c-86296b575d79 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:37.036936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 22/46] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jun 27 15:33:36 2022] POST /identity/v3/projects => generated 312 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:33:38.922327 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c078bc-7351-4de4-b22f-cb3e2e02955a None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:38.922830 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c078bc-7351-4de4-b22f-cb3e2e02955a None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:38.922931 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c078bc-7351-4de4-b22f-cb3e2e02955a None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:38.924281 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 25/47] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:38.929767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60766b-a510-4301-bedd-b858c83d525a None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:38.930163 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60766b-a510-4301-bedd-b858c83d525a None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:38.930370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60766b-a510-4301-bedd-b858c83d525a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:38.947644 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-9b60766b-a510-4301-bedd-b858c83d525a None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:38.948856 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-9b60766b-a510-4301-bedd-b858c83d525a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:39.001607 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:39.022931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 23/48] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:39.047284 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c078bc-7351-4de4-b22f-cb3e2e02955a None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:39.116409 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75eafb-68eb-4e1b-8647-2b0c84b66708 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.116697 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75eafb-68eb-4e1b-8647-2b0c84b66708 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.116976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75eafb-68eb-4e1b-8647-2b0c84b66708 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.118385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 24/49] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:33:39.123289 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80756b-bf3b-487e-8cbd-6d3076f81280 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.123573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80756b-bf3b-487e-8cbd-6d3076f81280 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.123851 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80756b-bf3b-487e-8cbd-6d3076f81280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.141081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-9c80756b-bf3b-487e-8cbd-6d3076f81280 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:39.142300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-9c80756b-bf3b-487e-8cbd-6d3076f81280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:39.151555 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c078bc-7351-4de4-b22f-cb3e2e02955a None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:39.186300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c667a4c-2246-433e-9305-c6c0a37605d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:39.187094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c667a4c-2246-433e-9305-c6c0a37605d9 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.187380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c667a4c-2246-433e-9305-c6c0a37605d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.187658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c667a4c-2246-433e-9305-c6c0a37605d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.198703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c667a4c-2246-433e-9305-c6c0a37605d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:33:39.206513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-8c667a4c-2246-433e-9305-c6c0a37605d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:33:39.206513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:39.206513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:39.206513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:39.206513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:39.206513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:39.206513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:39.206513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:39.206513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:39.206513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:39.206513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:39.206513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:39.206513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:33:39.206513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:33:39.206513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:33:39.206513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:33:39.207500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:33:39.207500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:33:39.207500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:33:39.207500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:33:39.207500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:39.207500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:39.207500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:39.207500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:39.207500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:39.207500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:39.207500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:39.207500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:39.207500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:39.207500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:39.207500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:39.207500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:39.208567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:39.208567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:39.208567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:39.208567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:39.208567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:33:39.208567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:33:39.208567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:33:39.208567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:33:39.208567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:33:39.208567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:33:39.208567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:33:39.208567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:33:39.208567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 26/50] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:33:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:33:39.211162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 25/51] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:33:39.213499 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c667a4c-2246-433e-9305-c6c0a37605d9 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:39.234194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80756b-bf3b-487e-8cbd-6d3076f81280 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:39.241050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:39.315707 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c667a4c-2246-433e-9305-c6c0a37605d9 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:39.338828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80756b-bf3b-487e-8cbd-6d3076f81280 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:39.355690 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba761d6-8b0e-4bb9-95f7-db294ff1235a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:39.356382 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba761d6-8b0e-4bb9-95f7-db294ff1235a None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.356661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba761d6-8b0e-4bb9-95f7-db294ff1235a None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.357426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba761d6-8b0e-4bb9-95f7-db294ff1235a None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.357562 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:39.357720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba761d6-8b0e-4bb9-95f7-db294ff1235a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:39.358032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba761d6-8b0e-4bb9-95f7-db294ff1235a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:39.359423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba761d6-8b0e-4bb9-95f7-db294ff1235a None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:39.368368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 27/52] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:33:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:39.373849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba761d6-8b0e-4bb9-95f7-db294ff1235a None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:39.378359 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b02bc-380f-4762-b669-30d6c5cb4804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:39.379159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b02bc-380f-4762-b669-30d6c5cb4804 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.379441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b02bc-380f-4762-b669-30d6c5cb4804 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.379716 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b02bc-380f-4762-b669-30d6c5cb4804 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.380899 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b02bc-380f-4762-b669-30d6c5cb4804 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:39.382425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b02bc-380f-4762-b669-30d6c5cb4804 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:39.384067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 26/53] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:33:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:33:39.389401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b02bc-380f-4762-b669-30d6c5cb4804 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:39.466030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:39.480275 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba761d6-8b0e-4bb9-95f7-db294ff1235a None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:39.494668 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b02bc-380f-4762-b669-30d6c5cb4804 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:39.513437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e393e28-6257-49b6-ab4c-7b9d55f73c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:39.514344 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e393e28-6257-49b6-ab4c-7b9d55f73c2e None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.514628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e393e28-6257-49b6-ab4c-7b9d55f73c2e None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.514907 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e393e28-6257-49b6-ab4c-7b9d55f73c2e None admin] PATH_INFO: `/v3/users/4b433ad0233c4e5dbe8fad9896962e25` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.516123 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e393e28-6257-49b6-ab4c-7b9d55f73c2e None admin] RBAC: Authorizing `identity:get_user(user_id=4b433ad0233c4e5dbe8fad9896962e25)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:39.517574 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e393e28-6257-49b6-ab4c-7b9d55f73c2e None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:39.519238 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 28/54] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:33:39 2022] GET /identity/v3/users/4b433ad0233c4e5dbe8fad9896962e25 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:33:39.525736 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e393e28-6257-49b6-ab4c-7b9d55f73c2e None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:39.530134 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea6532-4905-48b8-ad36-cf89647d522f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:39.530801 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea6532-4905-48b8-ad36-cf89647d522f None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.531089 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea6532-4905-48b8-ad36-cf89647d522f None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.531369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea6532-4905-48b8-ad36-cf89647d522f None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.532083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea6532-4905-48b8-ad36-cf89647d522f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:39.533532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea6532-4905-48b8-ad36-cf89647d522f None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:39.559697 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 27/55] 10.222.0.20 () {60 vars in 1140 bytes} [Mon Jun 27 15:33:39 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:33:39.634889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e393e28-6257-49b6-ab4c-7b9d55f73c2e None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:39.671716 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd8085-3cac-498e-b42c-d73f614824ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:39.672530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd8085-3cac-498e-b42c-d73f614824ef None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.672816 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd8085-3cac-498e-b42c-d73f614824ef None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.673094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd8085-3cac-498e-b42c-d73f614824ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.674454 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd8085-3cac-498e-b42c-d73f614824ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:39.675962 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd8085-3cac-498e-b42c-d73f614824ef None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:39.677705 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 29/56] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:33:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:33:39.683824 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea6532-4905-48b8-ad36-cf89647d522f None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:39.720909 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:39.789153 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea6532-4905-48b8-ad36-cf89647d522f None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:39.824750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200d535-ad5b-42d4-944a-435a8d28fc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:39.825238 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200d535-ad5b-42d4-944a-435a8d28fc98 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:39.825529 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200d535-ad5b-42d4-944a-435a8d28fc98 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:39.825843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200d535-ad5b-42d4-944a-435a8d28fc98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:39.826702 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3200d535-ad5b-42d4-944a-435a8d28fc98 None admin] RBAC: Adding query filter params (role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.domain.id=default, user.id=4b433ad0233c4e5dbe8fad9896962e25) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:39.827024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3200d535-ad5b-42d4-944a-435a8d28fc98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:39.828480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3200d535-ad5b-42d4-944a-435a8d28fc98 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:39.839059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 28/57] 10.222.0.20 () {56 vars in 1322 bytes} [Mon Jun 27 15:33:39 2022] GET /identity/v3/role_assignments?user.id=4b433ad0233c4e5dbe8fad9896962e25&scope.domain.id=default&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:41.612114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4fc70-549f-4646-b5d1-81b36c415cb5 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:41.612626 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4fc70-549f-4646-b5d1-81b36c415cb5 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:41.612736 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4fc70-549f-4646-b5d1-81b36c415cb5 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:41.614102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 30/58] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:41.619149 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102ef9d-57e7-4075-a0fc-f452348af2d3 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:41.619508 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102ef9d-57e7-4075-a0fc-f452348af2d3 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:41.619713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102ef9d-57e7-4075-a0fc-f452348af2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:41.636045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-b102ef9d-57e7-4075-a0fc-f452348af2d3 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:41.637227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-b102ef9d-57e7-4075-a0fc-f452348af2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:41.703194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 29/59] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:41.728951 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4fc70-549f-4646-b5d1-81b36c415cb5 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:41.755240 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:41.840025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4fc70-549f-4646-b5d1-81b36c415cb5 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:41.874044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352d529-4f03-4f04-8b38-589b51840b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:41.874767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352d529-4f03-4f04-8b38-589b51840b5d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:41.875055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352d529-4f03-4f04-8b38-589b51840b5d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:41.875339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352d529-4f03-4f04-8b38-589b51840b5d None admin] PATH_INFO: `/v3/domains/default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:41.876515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352d529-4f03-4f04-8b38-589b51840b5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:41.878003 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352d529-4f03-4f04-8b38-589b51840b5d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:41.879621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 31/60] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:33:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:41.886205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102ef9d-57e7-4075-a0fc-f452348af2d3 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:41.916630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:41.986101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b102ef9d-57e7-4075-a0fc-f452348af2d3 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:42.020327 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ae301-b151-4cc0-8276-4755fce00f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:42.020995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ae301-b151-4cc0-8276-4755fce00f17 None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:42.021286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ae301-b151-4cc0-8276-4755fce00f17 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:42.021568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ae301-b151-4cc0-8276-4755fce00f17 None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:42.022315 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ae301-b151-4cc0-8276-4755fce00f17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:42.023761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ae301-b151-4cc0-8276-4755fce00f17 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:42.054639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 30/61] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jun 27 15:33:41 2022] POST /identity/v3/users => generated 275 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:33:45.402125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf7688-5bfd-4f6f-8d1f-d9e47095ba76 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:45.402633 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf7688-5bfd-4f6f-8d1f-d9e47095ba76 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:45.402707 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf7688-5bfd-4f6f-8d1f-d9e47095ba76 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:45.404051 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 32/62] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:33:45.409468 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a3afe-5267-4d98-88ed-70b254d68d1a None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:45.409915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a3afe-5267-4d98-88ed-70b254d68d1a None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:45.410119 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a3afe-5267-4d98-88ed-70b254d68d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:45.434695 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-880a3afe-5267-4d98-88ed-70b254d68d1a None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:45.435927 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-880a3afe-5267-4d98-88ed-70b254d68d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:45.520996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049a8c2-ccb7-4d87-8b99-4e0ae232ac3e None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:45.521537 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049a8c2-ccb7-4d87-8b99-4e0ae232ac3e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:45.521978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049a8c2-ccb7-4d87-8b99-4e0ae232ac3e None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:45.523756 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 33/63] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:45.529963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ededd9-7d59-4937-ae4e-964918e9ea91 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:45.530484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ededd9-7d59-4937-ae4e-964918e9ea91 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:45.530900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ededd9-7d59-4937-ae4e-964918e9ea91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:45.537431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 31/64] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:45.561024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a3afe-5267-4d98-88ed-70b254d68d1a None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:45.565815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-46ededd9-7d59-4937-ae4e-964918e9ea91 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:45.572359 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-46ededd9-7d59-4937-ae4e-964918e9ea91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:45.588255 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:45.686101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a3afe-5267-4d98-88ed-70b254d68d1a None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:45.700921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 34/65] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 27 15:33:45.702837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d284bf-64cd-45d7-a452-aa4aec1415a0 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:45.703135 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d284bf-64cd-45d7-a452-aa4aec1415a0 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:45.703421 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d284bf-64cd-45d7-a452-aa4aec1415a0 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:45.705571 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 35/66] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:45.710780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f855a9-56b7-4fb4-a7e0-ad9fb598c021 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:45.711071 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f855a9-56b7-4fb4-a7e0-ad9fb598c021 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:45.711351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f855a9-56b7-4fb4-a7e0-ad9fb598c021 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:45.713882 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 36/67] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:45.715049 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a400685-1ae5-49f3-8cc4-7a87c98b04fd None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:45.715587 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a400685-1ae5-49f3-8cc4-7a87c98b04fd None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:45.716066 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a400685-1ae5-49f3-8cc4-7a87c98b04fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:45.734618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-2a400685-1ae5-49f3-8cc4-7a87c98b04fd None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:45.735829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-2a400685-1ae5-49f3-8cc4-7a87c98b04fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:45.736931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c67990-bdd6-4a2d-81b2-cf3ea129972f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:45.737774 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c67990-bdd6-4a2d-81b2-cf3ea129972f None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:45.738054 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c67990-bdd6-4a2d-81b2-cf3ea129972f None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:45.738480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c67990-bdd6-4a2d-81b2-cf3ea129972f None admin] PATH_INFO: `/v3/roles/member` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:45.750584 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c67990-bdd6-4a2d-81b2-cf3ea129972f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:33:45.755272 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:45.759259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-e5c67990-bdd6-4a2d-81b2-cf3ea129972f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:33:45.759259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:45.759259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:45.759259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:45.759259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:45.759259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:45.759259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:45.759259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:45.759259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:45.759259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:45.759259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:45.759259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:45.759259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:33:45.759259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:33:45.759259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:33:45.759259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:33:45.760116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:33:45.760116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:33:45.760116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:33:45.760116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:33:45.760116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:45.760116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:45.760116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:45.760116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:45.760116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:45.760116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:45.760116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:45.760116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:45.760116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:45.760116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:45.760116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:45.760116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:45.760839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:45.760839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:45.760839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:45.760839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:45.760839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:33:45.760839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:33:45.760839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:33:45.760839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:33:45.760839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:33:45.760839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:33:45.760839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:33:45.760839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:33:45.761417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 32/68] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:33:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:33:45.762624 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6906b3-4b02-4a2e-bb3d-b193fcccf985 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:45.762624 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6906b3-4b02-4a2e-bb3d-b193fcccf985 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:45.762780 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6906b3-4b02-4a2e-bb3d-b193fcccf985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:45.780782 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-bf6906b3-4b02-4a2e-bb3d-b193fcccf985 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:45.782030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-bf6906b3-4b02-4a2e-bb3d-b193fcccf985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:45.818148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 37/69] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:45.819286 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a400685-1ae5-49f3-8cc4-7a87c98b04fd None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:45.851935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 33/70] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:45.853924 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6906b3-4b02-4a2e-bb3d-b193fcccf985 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:45.862522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:45.935256 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a400685-1ae5-49f3-8cc4-7a87c98b04fd None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:45.955205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6906b3-4b02-4a2e-bb3d-b193fcccf985 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:45.971320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507202b-d637-47a2-aed8-71e03ddc7461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:45.972132 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507202b-d637-47a2-aed8-71e03ddc7461 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:45.972420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507202b-d637-47a2-aed8-71e03ddc7461 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:45.972710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507202b-d637-47a2-aed8-71e03ddc7461 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:45.983753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507202b-d637-47a2-aed8-71e03ddc7461 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:33:45.989902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905cb31-5280-4a0a-9383-8df4bc2684d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:45.990576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905cb31-5280-4a0a-9383-8df4bc2684d0 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:45.990862 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905cb31-5280-4a0a-9383-8df4bc2684d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:45.991141 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905cb31-5280-4a0a-9383-8df4bc2684d0 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:45.991893 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905cb31-5280-4a0a-9383-8df4bc2684d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:45.991975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-c507202b-d637-47a2-aed8-71e03ddc7461 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:33:45.991975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:45.991975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:45.991975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:45.991975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:45.991975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:45.991975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:45.991975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:45.991975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:45.991975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:45.991975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:45.991975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:45.991975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:33:45.991975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:33:45.991975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:33:45.991975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:33:45.992837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:33:45.992837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:33:45.992837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:33:45.992837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:33:45.992837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:45.992837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:45.992837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:45.992837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:45.992837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:45.992837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:45.992837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:45.992837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:45.992837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:45.992837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:45.992837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:45.992837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:45.993654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:45.993654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:45.993654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:45.993654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:45.993654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:33:45.993654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:33:45.993654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:33:45.993654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:33:45.993654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:33:45.993654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:33:45.993654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:33:45.993654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:33:45.994316 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905cb31-5280-4a0a-9383-8df4bc2684d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:45.994381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 38/71] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:33:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:33:45.994441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905cb31-5280-4a0a-9383-8df4bc2684d0 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:45.995177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b08fe-4f0c-4294-86f6-b38fb424bdeb None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:45.995478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b08fe-4f0c-4294-86f6-b38fb424bdeb None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:45.995764 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b08fe-4f0c-4294-86f6-b38fb424bdeb None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:45.997700 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 39/72] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:45.997821 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b08fe-4f0c-4294-86f6-b38fb424bdeb None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:46.009859 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 34/73] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:33:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:46.010778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905cb31-5280-4a0a-9383-8df4bc2684d0 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:46.016319 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:46.079741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:46.108117 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b08fe-4f0c-4294-86f6-b38fb424bdeb None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:46.114742 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905cb31-5280-4a0a-9383-8df4bc2684d0 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:46.142018 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55210966-81d4-4adc-bc94-46e41080b95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:46.142838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55210966-81d4-4adc-bc94-46e41080b95c None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:46.143125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55210966-81d4-4adc-bc94-46e41080b95c None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:46.143402 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55210966-81d4-4adc-bc94-46e41080b95c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:46.148969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824f7c3-cdf7-45fe-8481-e396b3be7b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:46.150304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824f7c3-cdf7-45fe-8481-e396b3be7b2f None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:46.150641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824f7c3-cdf7-45fe-8481-e396b3be7b2f None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:46.151074 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824f7c3-cdf7-45fe-8481-e396b3be7b2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:46.154208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55210966-81d4-4adc-bc94-46e41080b95c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:33:46.161746 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824f7c3-cdf7-45fe-8481-e396b3be7b2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:33:46.162492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-55210966-81d4-4adc-bc94-46e41080b95c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:33:46.162492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:46.162492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:46.162492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:46.162492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:46.162492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:46.162492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:46.162492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:46.162492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:46.162492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:46.162492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:46.162492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:46.162492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:33:46.162492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:33:46.162492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:33:46.162492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:33:46.163484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:33:46.163484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:33:46.163484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:33:46.163484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:33:46.163484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:46.163484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:46.163484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:46.163484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:46.163484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:46.163484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:46.163484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:46.163484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:46.163484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:46.163484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:46.163484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:46.163484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:46.164252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:46.164252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:46.164252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:46.164252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:46.164252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:33:46.164252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:33:46.164252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:33:46.164252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:33:46.164252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:33:46.164252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:33:46.164252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:33:46.164252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:33:46.164252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 40/74] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:33:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:33:46.164889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55210966-81d4-4adc-bc94-46e41080b95c None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:46.171473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-9824f7c3-cdf7-45fe-8481-e396b3be7b2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:33:46.171473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:46.171473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:46.171473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:46.171473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:46.171473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:46.171473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:46.171473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:46.171473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:46.171473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:46.171473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:46.171473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:46.171473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:33:46.171473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:33:46.171473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:33:46.171473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:33:46.172522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:33:46.172522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:33:46.172522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:33:46.172522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:33:46.172522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:46.172522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:46.172522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:46.172522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:46.172522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:46.172522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:46.172522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:46.172522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:46.172522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:46.172522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:46.172522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:46.172522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:46.173294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:46.173294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:46.173294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:46.173294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:46.173294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:33:46.173294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:33:46.173294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:33:46.173294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:33:46.173294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:33:46.173294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:33:46.173294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:33:46.173294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:33:46.173294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 35/75] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:33:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:33:46.173928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a79dc-bd1e-4c6e-8c9d-eca0b09b5106 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:46.174202 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a79dc-bd1e-4c6e-8c9d-eca0b09b5106 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:46.174480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a79dc-bd1e-4c6e-8c9d-eca0b09b5106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:46.191429 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-e15a79dc-bd1e-4c6e-8c9d-eca0b09b5106 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:46.192625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e15a79dc-bd1e-4c6e-8c9d-eca0b09b5106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:46.238495 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:46.260847 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 36/76] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:46.261728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a79dc-bd1e-4c6e-8c9d-eca0b09b5106 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:46.280172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55210966-81d4-4adc-bc94-46e41080b95c None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:46.315335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a87209-a5b7-4053-893a-5878cd5b0e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:46.316030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a87209-a5b7-4053-893a-5878cd5b0e7f None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:46.316320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a87209-a5b7-4053-893a-5878cd5b0e7f None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:46.316602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a87209-a5b7-4053-893a-5878cd5b0e7f None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:46.317360 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a87209-a5b7-4053-893a-5878cd5b0e7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:46.317705 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a87209-a5b7-4053-893a-5878cd5b0e7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:46.319119 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a87209-a5b7-4053-893a-5878cd5b0e7f None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:46.323595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:46.328007 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 41/77] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:33:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:46.328814 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a87209-a5b7-4053-893a-5878cd5b0e7f None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:46.365441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a79dc-bd1e-4c6e-8c9d-eca0b09b5106 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:46.400113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7fd17-5d6a-4971-a5e2-f0a0d899b228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:46.400991 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7fd17-5d6a-4971-a5e2-f0a0d899b228 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:46.401280 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7fd17-5d6a-4971-a5e2-f0a0d899b228 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:46.401559 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7fd17-5d6a-4971-a5e2-f0a0d899b228 None admin] PATH_INFO: `/v3/users/5c685740a045443997f807d9439f89c3` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:46.417416 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7fd17-5d6a-4971-a5e2-f0a0d899b228 None admin] RBAC: Authorizing `identity:get_user(user_id=5c685740a045443997f807d9439f89c3)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:46.418936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7fd17-5d6a-4971-a5e2-f0a0d899b228 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:46.420628 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 37/78] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:33:46 2022] GET /identity/v3/users/5c685740a045443997f807d9439f89c3 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:46.421385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7fd17-5d6a-4971-a5e2-f0a0d899b228 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:46.436695 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a87209-a5b7-4053-893a-5878cd5b0e7f None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:46.476572 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542662d-771b-441c-b6e2-adad829eb61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:46.477984 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542662d-771b-441c-b6e2-adad829eb61c None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:46.478438 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542662d-771b-441c-b6e2-adad829eb61c None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:46.478720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542662d-771b-441c-b6e2-adad829eb61c None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:46.479582 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542662d-771b-441c-b6e2-adad829eb61c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:46.479964 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542662d-771b-441c-b6e2-adad829eb61c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:46.481625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542662d-771b-441c-b6e2-adad829eb61c None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:46.483604 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:46.498247 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 42/79] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:33:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:46.498247 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542662d-771b-441c-b6e2-adad829eb61c None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:46.537462 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7fd17-5d6a-4971-a5e2-f0a0d899b228 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:46.558333 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:46.576591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43fe95-94c1-435d-8ccc-f3eae6356cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:46.577268 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43fe95-94c1-435d-8ccc-f3eae6356cb6 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:46.577555 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43fe95-94c1-435d-8ccc-f3eae6356cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:46.577871 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43fe95-94c1-435d-8ccc-f3eae6356cb6 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:46.578629 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43fe95-94c1-435d-8ccc-f3eae6356cb6 None admin] RBAC: Adding query filter params (name=member) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:46.578939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43fe95-94c1-435d-8ccc-f3eae6356cb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:46.580327 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43fe95-94c1-435d-8ccc-f3eae6356cb6 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:46.588891 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 38/80] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:33:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:46.589520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43fe95-94c1-435d-8ccc-f3eae6356cb6 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:46.605966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542662d-771b-441c-b6e2-adad829eb61c None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:46.640826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341c52c-75ce-4b31-ac9c-934ced8d7bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:46.641623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341c52c-75ce-4b31-ac9c-934ced8d7bff None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:46.641934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341c52c-75ce-4b31-ac9c-934ced8d7bff None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:46.642209 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341c52c-75ce-4b31-ac9c-934ced8d7bff None admin] PATH_INFO: `/v3/domains/default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:46.643389 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341c52c-75ce-4b31-ac9c-934ced8d7bff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:46.644849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341c52c-75ce-4b31-ac9c-934ced8d7bff None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:46.646489 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 43/81] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:33:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:46.651697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341c52c-75ce-4b31-ac9c-934ced8d7bff None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:46.690318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43fe95-94c1-435d-8ccc-f3eae6356cb6 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:46.713369 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:46.725200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b2cd8-9987-4a16-bfe0-b43a2da8118b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:46.726127 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b2cd8-9987-4a16-bfe0-b43a2da8118b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:46.726421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b2cd8-9987-4a16-bfe0-b43a2da8118b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:46.726704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b2cd8-9987-4a16-bfe0-b43a2da8118b None admin] PATH_INFO: `/v3/users/4b433ad0233c4e5dbe8fad9896962e25` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:46.728968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b2cd8-9987-4a16-bfe0-b43a2da8118b None admin] RBAC: Authorizing `identity:get_user(user_id=4b433ad0233c4e5dbe8fad9896962e25)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:46.730473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b2cd8-9987-4a16-bfe0-b43a2da8118b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:46.732143 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 39/82] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:33:46 2022] GET /identity/v3/users/4b433ad0233c4e5dbe8fad9896962e25 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:46.732904 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b2cd8-9987-4a16-bfe0-b43a2da8118b None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:46.753117 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341c52c-75ce-4b31-ac9c-934ced8d7bff None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:46.781401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:46.793937 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeac8ce-fc17-44c0-8585-92e8f2681c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:46.794604 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeac8ce-fc17-44c0-8585-92e8f2681c94 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:46.794902 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeac8ce-fc17-44c0-8585-92e8f2681c94 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:46.795186 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeac8ce-fc17-44c0-8585-92e8f2681c94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:46.839607 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b2cd8-9987-4a16-bfe0-b43a2da8118b None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:46.868642 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeac8ce-fc17-44c0-8585-92e8f2681c94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:46.870637 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeac8ce-fc17-44c0-8585-92e8f2681c94 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:46.874224 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8095f5-8fc3-4854-af81-39fc51e12816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:46.875105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8095f5-8fc3-4854-af81-39fc51e12816 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:46.875387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8095f5-8fc3-4854-af81-39fc51e12816 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:46.875660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8095f5-8fc3-4854-af81-39fc51e12816 None admin] PATH_INFO: `/v3/users/5c685740a045443997f807d9439f89c3` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:46.876878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8095f5-8fc3-4854-af81-39fc51e12816 None admin] RBAC: Authorizing `identity:get_user(user_id=5c685740a045443997f807d9439f89c3)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:46.878378 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8095f5-8fc3-4854-af81-39fc51e12816 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:46.880011 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 40/83] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:33:46 2022] GET /identity/v3/users/5c685740a045443997f807d9439f89c3 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:46.880764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8095f5-8fc3-4854-af81-39fc51e12816 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:46.939180 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:46.979817 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 44/84] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:33:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:33:46.980648 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeac8ce-fc17-44c0-8585-92e8f2681c94 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:46.988481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8095f5-8fc3-4854-af81-39fc51e12816 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:47.002790 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:47.022699 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774561d-15cd-4e9b-aa8b-d480fcaf4841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:47.023500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774561d-15cd-4e9b-aa8b-d480fcaf4841 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:47.023785 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774561d-15cd-4e9b-aa8b-d480fcaf4841 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:47.024063 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774561d-15cd-4e9b-aa8b-d480fcaf4841 None admin] PATH_INFO: `/v3/users/5c685740a045443997f807d9439f89c3` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:47.025808 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774561d-15cd-4e9b-aa8b-d480fcaf4841 None admin] RBAC: Authorizing `identity:get_user(user_id=5c685740a045443997f807d9439f89c3)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:47.027258 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774561d-15cd-4e9b-aa8b-d480fcaf4841 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:47.028887 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 41/85] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:33:46 2022] GET /identity/v3/users/5c685740a045443997f807d9439f89c3 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:47.029655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774561d-15cd-4e9b-aa8b-d480fcaf4841 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:47.090323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeac8ce-fc17-44c0-8585-92e8f2681c94 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:47.124595 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ce820-c635-4f36-962d-4b520c35afb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:47.125271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ce820-c635-4f36-962d-4b520c35afb9 None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:47.125562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ce820-c635-4f36-962d-4b520c35afb9 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:47.125867 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ce820-c635-4f36-962d-4b520c35afb9 None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:47.126584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ce820-c635-4f36-962d-4b520c35afb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:47.128047 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ce820-c635-4f36-962d-4b520c35afb9 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:47.135362 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774561d-15cd-4e9b-aa8b-d480fcaf4841 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:47.155567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 45/86] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jun 27 15:33:46 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:33:47.156421 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ce820-c635-4f36-962d-4b520c35afb9 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:47.178675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72366e9f-6b9d-4618-ad9b-f63737f61071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:47.179400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72366e9f-6b9d-4618-ad9b-f63737f61071 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:47.179687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72366e9f-6b9d-4618-ad9b-f63737f61071 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:47.179968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72366e9f-6b9d-4618-ad9b-f63737f61071 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:47.195259 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:47.240914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:47.263390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72366e9f-6b9d-4618-ad9b-f63737f61071 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:47.264928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72366e9f-6b9d-4618-ad9b-f63737f61071 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:47.297151 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ce820-c635-4f36-962d-4b520c35afb9 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:47.337422 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4b50b-f6ff-447e-bc2b-d75f54572751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:47.338128 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4b50b-f6ff-447e-bc2b-d75f54572751 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:47.338728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4b50b-f6ff-447e-bc2b-d75f54572751 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:47.338728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4b50b-f6ff-447e-bc2b-d75f54572751 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:47.394665 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 42/87] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:33:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:33:47.395527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72366e9f-6b9d-4618-ad9b-f63737f61071 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:47.426543 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4b50b-f6ff-447e-bc2b-d75f54572751 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:47.428050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4b50b-f6ff-447e-bc2b-d75f54572751 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:47.455513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:47.504415 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:47.528646 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72366e9f-6b9d-4618-ad9b-f63737f61071 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:47.554867 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 46/88] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:33:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:33:47.556569 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4b50b-f6ff-447e-bc2b-d75f54572751 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:47.578474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9833a7-7248-4894-b74c-2824616c7e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:47.579582 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9833a7-7248-4894-b74c-2824616c7e3c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:47.580013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9833a7-7248-4894-b74c-2824616c7e3c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:47.580433 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9833a7-7248-4894-b74c-2824616c7e3c None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:47.582006 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9833a7-7248-4894-b74c-2824616c7e3c None admin] RBAC: Authorizing `identity:get_project(project_id=c0f0739222694cedabb8f12e4ce82e9b)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:47.583618 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9833a7-7248-4894-b74c-2824616c7e3c None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:47.585485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 43/89] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:33:47 2022] GET /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:47.586478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9833a7-7248-4894-b74c-2824616c7e3c None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:47.685253 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4b50b-f6ff-447e-bc2b-d75f54572751 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:47.713660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9833a7-7248-4894-b74c-2824616c7e3c None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:47.715636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:47.735161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06e821-1552-45c3-8c09-bd18f8cb79c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:47.736217 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06e821-1552-45c3-8c09-bd18f8cb79c5 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:47.736512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06e821-1552-45c3-8c09-bd18f8cb79c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:47.736797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06e821-1552-45c3-8c09-bd18f8cb79c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:47.755449 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf2521-abbc-4a23-bf53-81aa08dc2153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:47.756328 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf2521-abbc-4a23-bf53-81aa08dc2153 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:47.756626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf2521-abbc-4a23-bf53-81aa08dc2153 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:47.756895 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf2521-abbc-4a23-bf53-81aa08dc2153 None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:47.758161 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf2521-abbc-4a23-bf53-81aa08dc2153 None admin] RBAC: Authorizing `identity:get_project(project_id=c0f0739222694cedabb8f12e4ce82e9b)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:47.760494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf2521-abbc-4a23-bf53-81aa08dc2153 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:47.762392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 44/90] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:33:47 2022] GET /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:47.763377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf2521-abbc-4a23-bf53-81aa08dc2153 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:47.796502 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:47.840253 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06e821-1552-45c3-8c09-bd18f8cb79c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:47.842263 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06e821-1552-45c3-8c09-bd18f8cb79c5 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:47.896305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf2521-abbc-4a23-bf53-81aa08dc2153 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:47.932705 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46788e10-9620-4844-8a1f-66aab34703be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:47.933585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46788e10-9620-4844-8a1f-66aab34703be None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:47.933919 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46788e10-9620-4844-8a1f-66aab34703be None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:47.934201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46788e10-9620-4844-8a1f-66aab34703be None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:47.935400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46788e10-9620-4844-8a1f-66aab34703be None admin] RBAC: Authorizing `identity:get_project(project_id=c0f0739222694cedabb8f12e4ce82e9b)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:47.936871 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46788e10-9620-4844-8a1f-66aab34703be None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:47.938652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 45/91] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:33:47 2022] GET /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:47.939482 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46788e10-9620-4844-8a1f-66aab34703be None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:47.962601 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 47/92] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:33:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:33:47.964109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06e821-1552-45c3-8c09-bd18f8cb79c5 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:47.988545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:48.039249 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:48.055611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46788e10-9620-4844-8a1f-66aab34703be None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:48.086391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06e821-1552-45c3-8c09-bd18f8cb79c5 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:48.091824 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083eee7-d0ef-47d0-be52-e5fd06d6f1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:48.092495 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083eee7-d0ef-47d0-be52-e5fd06d6f1e1 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:48.092783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083eee7-d0ef-47d0-be52-e5fd06d6f1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:48.093063 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083eee7-d0ef-47d0-be52-e5fd06d6f1e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:48.093939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083eee7-d0ef-47d0-be52-e5fd06d6f1e1 None admin] RBAC: Adding query filter params (role.id=5dc71eded4f64c08b2fb5f545b63b2f6, scope.project.id=c0f0739222694cedabb8f12e4ce82e9b, user.id=5c685740a045443997f807d9439f89c3) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:48.094255 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083eee7-d0ef-47d0-be52-e5fd06d6f1e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:48.095709 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083eee7-d0ef-47d0-be52-e5fd06d6f1e1 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:48.101589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 46/93] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:33:47 2022] GET /identity/v3/role_assignments?user.id=5c685740a045443997f807d9439f89c3&scope.project.id=c0f0739222694cedabb8f12e4ce82e9b&role.id=5dc71eded4f64c08b2fb5f545b63b2f6 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:48.102511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083eee7-d0ef-47d0-be52-e5fd06d6f1e1 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:48.129936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614cade-cbcd-433f-bb17-fe7f771b9ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:48.131095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614cade-cbcd-433f-bb17-fe7f771b9ec3 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:48.131592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614cade-cbcd-433f-bb17-fe7f771b9ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:48.132131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614cade-cbcd-433f-bb17-fe7f771b9ec3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:48.133417 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614cade-cbcd-433f-bb17-fe7f771b9ec3 None admin] RBAC: Adding query filter params (role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.project.id=c0f0739222694cedabb8f12e4ce82e9b, user.id=4b433ad0233c4e5dbe8fad9896962e25) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:48.134634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614cade-cbcd-433f-bb17-fe7f771b9ec3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:48.136631 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614cade-cbcd-433f-bb17-fe7f771b9ec3 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:48.143521 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 48/94] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:33:47 2022] GET /identity/v3/role_assignments?user.id=4b433ad0233c4e5dbe8fad9896962e25&scope.project.id=c0f0739222694cedabb8f12e4ce82e9b&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:48.144770 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614cade-cbcd-433f-bb17-fe7f771b9ec3 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:48.215287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083eee7-d0ef-47d0-be52-e5fd06d6f1e1 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:48.250721 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:48.253783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04e7c5-7c55-4797-9670-26e1c1d6562f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:48.254713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04e7c5-7c55-4797-9670-26e1c1d6562f None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:48.254982 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04e7c5-7c55-4797-9670-26e1c1d6562f None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:48.255382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04e7c5-7c55-4797-9670-26e1c1d6562f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:48.256149 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04e7c5-7c55-4797-9670-26e1c1d6562f None admin] RBAC: Adding query filter params (role.id=478b9cd589e846588025a4b4ed2c07f0, scope.project.id=c0f0739222694cedabb8f12e4ce82e9b, user.id=5c685740a045443997f807d9439f89c3) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:48.256474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04e7c5-7c55-4797-9670-26e1c1d6562f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:48.258056 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04e7c5-7c55-4797-9670-26e1c1d6562f None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:48.265116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 47/95] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:33:48 2022] GET /identity/v3/role_assignments?user.id=5c685740a045443997f807d9439f89c3&scope.project.id=c0f0739222694cedabb8f12e4ce82e9b&role.id=478b9cd589e846588025a4b4ed2c07f0 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:48.275638 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614cade-cbcd-433f-bb17-fe7f771b9ec3 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:48.322000 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff58ce0-f4a5-43c1-8bdb-918dbee83b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:48.323384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff58ce0-f4a5-43c1-8bdb-918dbee83b27 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:48.323967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff58ce0-f4a5-43c1-8bdb-918dbee83b27 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:48.324530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff58ce0-f4a5-43c1-8bdb-918dbee83b27 None admin] PATH_INFO: `/v3/projects/1b1e8d351c924a2ab852be2c298ce115` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:48.326722 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff58ce0-f4a5-43c1-8bdb-918dbee83b27 None admin] RBAC: Authorizing `identity:get_project(project_id=1b1e8d351c924a2ab852be2c298ce115)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:48.328885 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff58ce0-f4a5-43c1-8bdb-918dbee83b27 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:48.331811 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 49/96] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:33:48 2022] GET /identity/v3/projects/1b1e8d351c924a2ab852be2c298ce115 => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:48.338908 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04e7c5-7c55-4797-9670-26e1c1d6562f None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:48.371424 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:48.497591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04e7c5-7c55-4797-9670-26e1c1d6562f None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:48.538536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09952b38-49c2-496e-93ef-9779d457a94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:48.539319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09952b38-49c2-496e-93ef-9779d457a94b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:48.539520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09952b38-49c2-496e-93ef-9779d457a94b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:48.539803 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09952b38-49c2-496e-93ef-9779d457a94b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:48.541711 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09952b38-49c2-496e-93ef-9779d457a94b None admin] RBAC: Adding query filter params (role.id=5dc71eded4f64c08b2fb5f545b63b2f6, scope.project.id=1b1e8d351c924a2ab852be2c298ce115, user.id=5c685740a045443997f807d9439f89c3) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:48.542040 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09952b38-49c2-496e-93ef-9779d457a94b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:48.543508 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09952b38-49c2-496e-93ef-9779d457a94b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:48.550145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 48/97] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:33:48 2022] GET /identity/v3/role_assignments?user.id=5c685740a045443997f807d9439f89c3&scope.project.id=1b1e8d351c924a2ab852be2c298ce115&role.id=5dc71eded4f64c08b2fb5f545b63b2f6 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:49.819501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff35eb-3503-423b-82d8-5c5bc32a7527 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:49.820031 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff35eb-3503-423b-82d8-5c5bc32a7527 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:49.820301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff35eb-3503-423b-82d8-5c5bc32a7527 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:49.822364 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 50/98] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:49.832081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe9883-c3a1-453b-96b4-54699bdaa950 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:49.832954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe9883-c3a1-453b-96b4-54699bdaa950 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:49.833446 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe9883-c3a1-453b-96b4-54699bdaa950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:49.851827 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-d8fe9883-c3a1-453b-96b4-54699bdaa950 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:49.854549 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-d8fe9883-c3a1-453b-96b4-54699bdaa950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:49.928097 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 49/99] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:49.954453 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff35eb-3503-423b-82d8-5c5bc32a7527 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:49.984020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b301cc-68ad-46c3-9db3-3facf37f50a0 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:49.984660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b301cc-68ad-46c3-9db3-3facf37f50a0 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:49.985157 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b301cc-68ad-46c3-9db3-3facf37f50a0 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:49.987189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 50/100] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:49.992480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e8c66-c975-4cb3-abe7-ea78f03981a9 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:49.993762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e8c66-c975-4cb3-abe7-ea78f03981a9 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:49.994514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e8c66-c975-4cb3-abe7-ea78f03981a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:50.020497 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-ab2e8c66-c975-4cb3-abe7-ea78f03981a9 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:50.022606 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-ab2e8c66-c975-4cb3-abe7-ea78f03981a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:50.028964 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:50.147010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 51/101] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:33:50.159496 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff35eb-3503-423b-82d8-5c5bc32a7527 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:50.207580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e8c66-c975-4cb3-abe7-ea78f03981a9 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:50.230535 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f17b4-7baa-47cb-bacb-0e2562e54de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:50.231375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f17b4-7baa-47cb-bacb-0e2562e54de7 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:50.231662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f17b4-7baa-47cb-bacb-0e2562e54de7 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:50.231945 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f17b4-7baa-47cb-bacb-0e2562e54de7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:50.233251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f17b4-7baa-47cb-bacb-0e2562e54de7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:50.234747 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f17b4-7baa-47cb-bacb-0e2562e54de7 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:50.236396 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 51/102] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:33:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:50.246949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f17b4-7baa-47cb-bacb-0e2562e54de7 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:50.298434 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:50.397391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f17b4-7baa-47cb-bacb-0e2562e54de7 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:50.433861 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9119cd-ce0b-4a14-9595-7c922db55352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:50.434532 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9119cd-ce0b-4a14-9595-7c922db55352 None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:50.434815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9119cd-ce0b-4a14-9595-7c922db55352 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:50.435091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9119cd-ce0b-4a14-9595-7c922db55352 None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:50.435810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9119cd-ce0b-4a14-9595-7c922db55352 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:50.437262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9119cd-ce0b-4a14-9595-7c922db55352 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:50.451191 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e8c66-c975-4cb3-abe7-ea78f03981a9 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:50.483198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 52/103] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jun 27 15:33:50 2022] POST /identity/v3/projects => generated 302 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:33:50.505958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afd4c0-39e2-4ba3-add3-075968d699f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:50.506810 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afd4c0-39e2-4ba3-add3-075968d699f4 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:50.507097 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afd4c0-39e2-4ba3-add3-075968d699f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:50.507428 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afd4c0-39e2-4ba3-add3-075968d699f4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:50.518976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:50.520301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afd4c0-39e2-4ba3-add3-075968d699f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:33:50.529962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-a7afd4c0-39e2-4ba3-add3-075968d699f4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:33:50.529962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:50.529962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:50.529962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:50.529962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:50.529962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:50.529962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:50.529962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:50.529962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:50.529962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:50.529962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:50.529962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:50.529962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:33:50.529962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:33:50.529962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:33:50.529962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:33:50.530918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:33:50.530918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:33:50.530918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:33:50.530918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:33:50.530918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:50.530918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:50.530918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:50.530918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:50.530918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:50.530918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:50.530918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:50.530918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:50.530918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:50.530918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:50.530918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:50.530918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:50.531703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:50.531703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:50.531703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:50.531703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:50.531703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:33:50.531703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:33:50.531703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:33:50.531703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:33:50.531703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:33:50.531703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:33:50.531703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:33:50.531703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:33:50.534485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 52/104] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:33:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:33:50.542788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9119cd-ce0b-4a14-9595-7c922db55352 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:50.624716 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:50.657736 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9119cd-ce0b-4a14-9595-7c922db55352 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:50.694768 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b413f5-cbc6-4bd7-8ffb-0d9245032b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:50.695458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b413f5-cbc6-4bd7-8ffb-0d9245032b1d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:50.695745 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b413f5-cbc6-4bd7-8ffb-0d9245032b1d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:50.696029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b413f5-cbc6-4bd7-8ffb-0d9245032b1d None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:50.697744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b413f5-cbc6-4bd7-8ffb-0d9245032b1d None admin] RBAC: Adding query filter params (name=reader) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:50.698065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b413f5-cbc6-4bd7-8ffb-0d9245032b1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:50.699488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b413f5-cbc6-4bd7-8ffb-0d9245032b1d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:50.708699 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 53/105] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:33:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:50.715000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afd4c0-39e2-4ba3-add3-075968d699f4 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:50.857064 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afd4c0-39e2-4ba3-add3-075968d699f4 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:50.905963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7bd91-afd1-4340-84de-73d1c2efc05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:50.906848 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7bd91-afd1-4340-84de-73d1c2efc05b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:50.907131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7bd91-afd1-4340-84de-73d1c2efc05b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:50.907405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7bd91-afd1-4340-84de-73d1c2efc05b None admin] PATH_INFO: `/v3/users/67655b82ae8f46caaa6337fa70edd15b` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:50.933257 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7bd91-afd1-4340-84de-73d1c2efc05b None admin] RBAC: Authorizing `identity:get_user(user_id=67655b82ae8f46caaa6337fa70edd15b)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:50.934781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7bd91-afd1-4340-84de-73d1c2efc05b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:50.936774 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 53/106] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:33:50 2022] GET /identity/v3/users/67655b82ae8f46caaa6337fa70edd15b => generated 289 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:50.942791 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b413f5-cbc6-4bd7-8ffb-0d9245032b1d None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:51.118851 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b413f5-cbc6-4bd7-8ffb-0d9245032b1d None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:51.172829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d7532-856d-4b5e-aacb-f94fe498468c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:51.173932 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d7532-856d-4b5e-aacb-f94fe498468c None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:51.174441 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d7532-856d-4b5e-aacb-f94fe498468c None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:51.175003 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d7532-856d-4b5e-aacb-f94fe498468c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:51.185898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:51.291802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d7532-856d-4b5e-aacb-f94fe498468c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:51.293707 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d7532-856d-4b5e-aacb-f94fe498468c None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:51.423258 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b459b24-1689-43c8-bfba-e9efdab8908c None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:51.423913 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b459b24-1689-43c8-bfba-e9efdab8908c None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:51.424404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b459b24-1689-43c8-bfba-e9efdab8908c None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:51.426194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 54/107] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:51.433764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448289a-95da-4452-8cf3-9049fd1c6036 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:51.434383 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448289a-95da-4452-8cf3-9049fd1c6036 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:51.434861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448289a-95da-4452-8cf3-9049fd1c6036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:51.441728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 54/108] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:33:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:33:51.443166 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d7532-856d-4b5e-aacb-f94fe498468c None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:51.480762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-3448289a-95da-4452-8cf3-9049fd1c6036 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:51.482452 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-3448289a-95da-4452-8cf3-9049fd1c6036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:51.530787 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:51.608875 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:51.648879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 55/109] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:51.650942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5cb55-941b-4ebb-a97a-4ac4c86a591d None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:51.651182 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5cb55-941b-4ebb-a97a-4ac4c86a591d None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:51.651450 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5cb55-941b-4ebb-a97a-4ac4c86a591d None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:51.652750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 56/110] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:51.658636 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a5210-b972-42f8-bfd1-8ae50466f917 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:51.659048 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a5210-b972-42f8-bfd1-8ae50466f917 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:51.659454 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a5210-b972-42f8-bfd1-8ae50466f917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:51.698412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-534a5210-b972-42f8-bfd1-8ae50466f917 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:51.700440 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d7532-856d-4b5e-aacb-f94fe498468c None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:51.701096 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-534a5210-b972-42f8-bfd1-8ae50466f917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:51.773281 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e06f6c-dcad-455d-b0df-25f72bad388f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:51.774171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e06f6c-dcad-455d-b0df-25f72bad388f None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:51.774490 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e06f6c-dcad-455d-b0df-25f72bad388f None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:51.774957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e06f6c-dcad-455d-b0df-25f72bad388f None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:51.776245 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e06f6c-dcad-455d-b0df-25f72bad388f None admin] RBAC: Authorizing `identity:get_project(project_id=c0f0739222694cedabb8f12e4ce82e9b)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:51.777755 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e06f6c-dcad-455d-b0df-25f72bad388f None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:51.779613 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 55/111] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:33:51 2022] GET /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b => generated 298 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:51.780636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e06f6c-dcad-455d-b0df-25f72bad388f None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:51.802172 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 57/112] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:51.804046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a32cb9-f7cf-4630-ac0b-6ebbf7d6454f None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:51.804558 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a32cb9-f7cf-4630-ac0b-6ebbf7d6454f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:51.805006 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a32cb9-f7cf-4630-ac0b-6ebbf7d6454f None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:51.807143 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 58/113] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:51.808274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625f7ce-e39b-4fe5-b8b5-1db069eedfdf None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:51.808574 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625f7ce-e39b-4fe5-b8b5-1db069eedfdf None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:51.808859 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625f7ce-e39b-4fe5-b8b5-1db069eedfdf None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:51.810424 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 59/114] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:51.811389 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625f7ce-e39b-4fe5-b8b5-1db069eedfdf None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:51.938465 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e06f6c-dcad-455d-b0df-25f72bad388f None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:51.941207 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:51.942151 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625f7ce-e39b-4fe5-b8b5-1db069eedfdf None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:51.964165 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:51.987809 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126c10c-458f-4abc-998e-6d17ff09671b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:51.989763 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126c10c-458f-4abc-998e-6d17ff09671b None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:51.990305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126c10c-458f-4abc-998e-6d17ff09671b None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:51.990766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126c10c-458f-4abc-998e-6d17ff09671b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:51.993251 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f640d8d-6454-4976-9a51-9798706f821c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:51.995018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f640d8d-6454-4976-9a51-9798706f821c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:51.995309 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f640d8d-6454-4976-9a51-9798706f821c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:51.995592 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f640d8d-6454-4976-9a51-9798706f821c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:51.996443 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f640d8d-6454-4976-9a51-9798706f821c None admin] RBAC: Adding query filter params (role.id=7a691f6f838a4a25a074d1537bb677a6, scope.project.id=c0f0739222694cedabb8f12e4ce82e9b, user.id=67655b82ae8f46caaa6337fa70edd15b) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:51.996766 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f640d8d-6454-4976-9a51-9798706f821c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:51.998270 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f640d8d-6454-4976-9a51-9798706f821c None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:52.004010 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5126c10c-458f-4abc-998e-6d17ff09671b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:33:52.014024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 60/115] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:33:51 2022] GET /identity/v3/role_assignments?user.id=67655b82ae8f46caaa6337fa70edd15b&scope.project.id=c0f0739222694cedabb8f12e4ce82e9b&role.id=7a691f6f838a4a25a074d1537bb677a6 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:52.014024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0f2a8-86f6-490c-8dcb-89977daeb74a None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.014024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0f2a8-86f6-490c-8dcb-89977daeb74a None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.014024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0f2a8-86f6-490c-8dcb-89977daeb74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.015070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-5126c10c-458f-4abc-998e-6d17ff09671b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:33:52.015070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:52.015070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:52.015070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:52.015070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:52.015070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:52.015070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:52.015070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:52.015070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:52.015070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:52.015070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:52.015070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:52.015070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:33:52.015070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:33:52.015070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:33:52.015070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:33:52.015878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:33:52.015878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:33:52.015878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:33:52.015878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:33:52.015878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:52.015878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:52.015878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:52.015878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:52.015878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:52.015878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:52.015878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:52.015878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:52.015878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:52.015878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:52.015878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:52.015878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:52.016619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:52.016619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:52.016619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:52.016619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:52.016619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:33:52.016619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:33:52.016619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:33:52.016619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:33:52.016619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:33:52.016619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:33:52.016619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:33:52.016619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:33:52.017187 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 56/116] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:33:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:33:52.019543 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e83e6-8d6d-4918-be88-31f506dac744 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.020087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e83e6-8d6d-4918-be88-31f506dac744 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.020603 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e83e6-8d6d-4918-be88-31f506dac744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.039939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-40c0f2a8-86f6-490c-8dcb-89977daeb74a None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:52.040770 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-698e83e6-8d6d-4918-be88-31f506dac744 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:52.042334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-698e83e6-8d6d-4918-be88-31f506dac744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:52.042566 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-40c0f2a8-86f6-490c-8dcb-89977daeb74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:52.150713 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 57/117] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:52.151989 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e83e6-8d6d-4918-be88-31f506dac744 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:52.174997 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 61/118] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:52.177161 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0f2a8-86f6-490c-8dcb-89977daeb74a None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:52.294175 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:52.306733 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e83e6-8d6d-4918-be88-31f506dac744 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:52.328022 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0f2a8-86f6-490c-8dcb-89977daeb74a None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:52.346147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:52.355072 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d4ec8-7444-4dcc-a44f-8ab79dd09654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:52.355988 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d4ec8-7444-4dcc-a44f-8ab79dd09654 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.356350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d4ec8-7444-4dcc-a44f-8ab79dd09654 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.356719 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d4ec8-7444-4dcc-a44f-8ab79dd09654 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.374934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d4ec8-7444-4dcc-a44f-8ab79dd09654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:33:52.378585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901da771-6a7c-473c-9ad2-732c2cc9e0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:52.379409 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901da771-6a7c-473c-9ad2-732c2cc9e0de None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.379799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901da771-6a7c-473c-9ad2-732c2cc9e0de None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.380181 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901da771-6a7c-473c-9ad2-732c2cc9e0de None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.381045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901da771-6a7c-473c-9ad2-732c2cc9e0de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:52.381610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901da771-6a7c-473c-9ad2-732c2cc9e0de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:52.383121 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901da771-6a7c-473c-9ad2-732c2cc9e0de None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:52.386977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-777d4ec8-7444-4dcc-a44f-8ab79dd09654 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:33:52.386977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:52.386977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:52.386977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:52.386977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:52.386977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:52.386977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:52.386977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:52.386977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:52.386977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:52.386977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:52.386977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:52.386977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:33:52.386977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:33:52.386977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:33:52.386977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:33:52.388040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:33:52.388040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:33:52.388040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:33:52.388040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:33:52.388040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:52.388040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:52.388040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:52.388040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:52.388040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:52.388040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:52.388040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:52.388040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:52.388040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:52.388040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:52.388040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:52.388040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:52.389043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:52.389043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:52.389043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:52.389043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:52.389043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:33:52.389043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:33:52.389043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:33:52.389043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:33:52.389043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:33:52.389043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:33:52.389043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:33:52.389043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:33:52.391251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 58/119] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:33:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:33:52.392273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d4ec8-7444-4dcc-a44f-8ab79dd09654 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:52.399018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 62/120] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:33:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:52.405163 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901da771-6a7c-473c-9ad2-732c2cc9e0de None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:52.518040 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901da771-6a7c-473c-9ad2-732c2cc9e0de None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:52.542153 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d4ec8-7444-4dcc-a44f-8ab79dd09654 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:52.558916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357c11c-00dd-4ba9-aba6-f34254d63575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:52.559645 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357c11c-00dd-4ba9-aba6-f34254d63575 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.559942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357c11c-00dd-4ba9-aba6-f34254d63575 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.560228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357c11c-00dd-4ba9-aba6-f34254d63575 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.571408 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357c11c-00dd-4ba9-aba6-f34254d63575 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:33:52.579485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-8357c11c-00dd-4ba9-aba6-f34254d63575 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:33:52.579485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:52.579485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:52.579485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:52.579485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:52.579485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:52.579485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:52.579485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:52.579485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:52.579485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:52.579485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:52.579485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:52.579485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:33:52.579485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:33:52.579485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:33:52.579485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:33:52.580372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:33:52.580372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:33:52.580372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:33:52.580372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:33:52.580372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:52.580372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:52.580372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:52.580372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:52.580372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:52.580372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:52.580372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:52.580372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:52.580372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:52.580372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:52.580372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:52.580372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:52.581115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:52.581115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:52.581115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:52.581115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:52.581115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:33:52.581115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:33:52.581115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:33:52.581115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:33:52.581115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:33:52.581115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:33:52.581115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:33:52.581115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:33:52.585890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 63/121] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:33:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:33:52.586860 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357c11c-00dd-4ba9-aba6-f34254d63575 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:52.589675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9839b-74bf-4159-bfe7-987bc4076ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:52.590991 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9839b-74bf-4159-bfe7-987bc4076ecb None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.592407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9839b-74bf-4159-bfe7-987bc4076ecb None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.592866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9839b-74bf-4159-bfe7-987bc4076ecb None admin] PATH_INFO: `/v3/roles/member` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.607942 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9839b-74bf-4159-bfe7-987bc4076ecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:33:52.620182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-06a9839b-74bf-4159-bfe7-987bc4076ecb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:33:52.620182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:52.620182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:52.620182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:52.620182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:52.620182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:52.620182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:52.620182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:52.620182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:52.620182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:52.620182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:52.620182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:52.620182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:33:52.620182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:33:52.620182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:33:52.620182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:33:52.621219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:33:52.621219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:33:52.621219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:33:52.621219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:33:52.621219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:52.621219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:52.621219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:52.621219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:52.621219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:52.621219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:52.621219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:52.621219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:52.621219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:52.621219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:52.621219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:52.621219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:52.622083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:52.622083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:52.622083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:52.622083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:52.622083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:33:52.622083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:33:52.622083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:33:52.622083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:33:52.622083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:33:52.622083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:33:52.622083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:33:52.622083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:33:52.623089 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 59/122] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:33:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:33:52.623905 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9839b-74bf-4159-bfe7-987bc4076ecb None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:52.627525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:52.643932 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:52.709795 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357c11c-00dd-4ba9-aba6-f34254d63575 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:52.742643 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9839b-74bf-4159-bfe7-987bc4076ecb None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:52.748184 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925ca10-7079-4ebe-b5cf-40eb3a113cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:52.749123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925ca10-7079-4ebe-b5cf-40eb3a113cf0 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.749649 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925ca10-7079-4ebe-b5cf-40eb3a113cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.750150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925ca10-7079-4ebe-b5cf-40eb3a113cf0 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.751167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925ca10-7079-4ebe-b5cf-40eb3a113cf0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:52.751680 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925ca10-7079-4ebe-b5cf-40eb3a113cf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:52.753405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925ca10-7079-4ebe-b5cf-40eb3a113cf0 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:52.763407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 64/123] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:33:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:52.764611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925ca10-7079-4ebe-b5cf-40eb3a113cf0 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:52.785359 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b97c22-e20a-4e58-8cbd-800ba6eeeefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:52.786708 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b97c22-e20a-4e58-8cbd-800ba6eeeefd None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.787004 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b97c22-e20a-4e58-8cbd-800ba6eeeefd None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.787334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b97c22-e20a-4e58-8cbd-800ba6eeeefd None admin] PATH_INFO: `/v3/users/5c685740a045443997f807d9439f89c3` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.788640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b97c22-e20a-4e58-8cbd-800ba6eeeefd None admin] RBAC: Authorizing `identity:get_user(user_id=5c685740a045443997f807d9439f89c3)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:52.790399 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b97c22-e20a-4e58-8cbd-800ba6eeeefd None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:52.792353 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 60/124] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:33:52 2022] GET /identity/v3/users/5c685740a045443997f807d9439f89c3 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:52.793121 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b97c22-e20a-4e58-8cbd-800ba6eeeefd None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:52.871299 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:52.880329 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925ca10-7079-4ebe-b5cf-40eb3a113cf0 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:52.895716 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:52.917717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b97c22-e20a-4e58-8cbd-800ba6eeeefd None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:52.920619 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd3cf9-377f-4771-b25f-39f71c0d1280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:52.921558 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd3cf9-377f-4771-b25f-39f71c0d1280 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.922090 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd3cf9-377f-4771-b25f-39f71c0d1280 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.922586 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd3cf9-377f-4771-b25f-39f71c0d1280 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.923541 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd3cf9-377f-4771-b25f-39f71c0d1280 None admin] RBAC: Adding query filter params (name=member) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:52.924052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd3cf9-377f-4771-b25f-39f71c0d1280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:52.925689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd3cf9-377f-4771-b25f-39f71c0d1280 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:52.935694 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 65/125] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:33:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:52.937026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd3cf9-377f-4771-b25f-39f71c0d1280 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:52.956949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb566a-b9c3-494a-b6f6-323b70dca012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:52.957704 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb566a-b9c3-494a-b6f6-323b70dca012 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:52.957998 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb566a-b9c3-494a-b6f6-323b70dca012 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:52.958275 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb566a-b9c3-494a-b6f6-323b70dca012 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:52.959081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb566a-b9c3-494a-b6f6-323b70dca012 None admin] RBAC: Adding query filter params (name=member) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:52.959390 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb566a-b9c3-494a-b6f6-323b70dca012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:52.961041 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb566a-b9c3-494a-b6f6-323b70dca012 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:52.971490 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 61/126] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:33:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:52.972478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb566a-b9c3-494a-b6f6-323b70dca012 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:53.050010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd3cf9-377f-4771-b25f-39f71c0d1280 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:53.086789 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb566a-b9c3-494a-b6f6-323b70dca012 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:53.089388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134333c-d1aa-4de2-a5af-403d96e73a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:53.090598 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134333c-d1aa-4de2-a5af-403d96e73a35 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.091203 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134333c-d1aa-4de2-a5af-403d96e73a35 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.091679 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134333c-d1aa-4de2-a5af-403d96e73a35 None admin] PATH_INFO: `/v3/users/4b433ad0233c4e5dbe8fad9896962e25` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.093109 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134333c-d1aa-4de2-a5af-403d96e73a35 None admin] RBAC: Authorizing `identity:get_user(user_id=4b433ad0233c4e5dbe8fad9896962e25)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:53.094953 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134333c-d1aa-4de2-a5af-403d96e73a35 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:53.097292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 66/127] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:33:52 2022] GET /identity/v3/users/4b433ad0233c4e5dbe8fad9896962e25 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:53.098388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134333c-d1aa-4de2-a5af-403d96e73a35 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:53.127687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:53.131098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3816e88-fcad-475d-965e-fec1180531ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:53.132207 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3816e88-fcad-475d-965e-fec1180531ae None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.132854 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3816e88-fcad-475d-965e-fec1180531ae None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.133142 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3816e88-fcad-475d-965e-fec1180531ae None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.134548 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3816e88-fcad-475d-965e-fec1180531ae None admin] RBAC: Authorizing `identity:get_project(project_id=c0f0739222694cedabb8f12e4ce82e9b)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:53.136172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3816e88-fcad-475d-965e-fec1180531ae None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:53.138190 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 62/128] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:33:52 2022] GET /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:53.138948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3816e88-fcad-475d-965e-fec1180531ae None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:53.152657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:53.218108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134333c-d1aa-4de2-a5af-403d96e73a35 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:53.253844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3816e88-fcad-475d-965e-fec1180531ae None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:53.257117 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01cc593-350a-4400-9e7a-8a7240b176d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:53.258244 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01cc593-350a-4400-9e7a-8a7240b176d2 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.258894 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01cc593-350a-4400-9e7a-8a7240b176d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.259505 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01cc593-350a-4400-9e7a-8a7240b176d2 None admin] PATH_INFO: `/v3/users/5c685740a045443997f807d9439f89c3` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.260951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01cc593-350a-4400-9e7a-8a7240b176d2 None admin] RBAC: Authorizing `identity:get_user(user_id=5c685740a045443997f807d9439f89c3)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:53.262708 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01cc593-350a-4400-9e7a-8a7240b176d2 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:53.264784 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 67/129] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:33:53 2022] GET /identity/v3/users/5c685740a045443997f807d9439f89c3 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:53.265950 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01cc593-350a-4400-9e7a-8a7240b176d2 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:53.292165 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e6cc7-3c6f-4f6d-a833-59d05ed725c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:53.293050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e6cc7-3c6f-4f6d-a833-59d05ed725c9 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.293335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e6cc7-3c6f-4f6d-a833-59d05ed725c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.293612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e6cc7-3c6f-4f6d-a833-59d05ed725c9 None admin] PATH_INFO: `/v3/users/5c685740a045443997f807d9439f89c3` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.294862 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e6cc7-3c6f-4f6d-a833-59d05ed725c9 None admin] RBAC: Authorizing `identity:get_user(user_id=5c685740a045443997f807d9439f89c3)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:53.296320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e6cc7-3c6f-4f6d-a833-59d05ed725c9 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:53.298208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 63/130] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:33:53 2022] GET /identity/v3/users/5c685740a045443997f807d9439f89c3 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:53.298942 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e6cc7-3c6f-4f6d-a833-59d05ed725c9 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:53.375035 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:53.381173 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:53.389712 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01cc593-350a-4400-9e7a-8a7240b176d2 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:53.404197 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e6cc7-3c6f-4f6d-a833-59d05ed725c9 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:53.428662 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897574b0-2085-4b7d-aa6d-82d807c05e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:53.429919 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897574b0-2085-4b7d-aa6d-82d807c05e44 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.430435 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897574b0-2085-4b7d-aa6d-82d807c05e44 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.431596 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897574b0-2085-4b7d-aa6d-82d807c05e44 None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.433144 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897574b0-2085-4b7d-aa6d-82d807c05e44 None admin] RBAC: Authorizing `identity:get_project(project_id=c0f0739222694cedabb8f12e4ce82e9b)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:53.434937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897574b0-2085-4b7d-aa6d-82d807c05e44 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:53.437083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 68/131] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:33:53 2022] GET /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:53.438218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897574b0-2085-4b7d-aa6d-82d807c05e44 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:53.445495 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c380cc-b900-4d03-9e68-48dfd5ba6c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:53.446531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c380cc-b900-4d03-9e68-48dfd5ba6c0d None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.446818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c380cc-b900-4d03-9e68-48dfd5ba6c0d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.447098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c380cc-b900-4d03-9e68-48dfd5ba6c0d None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b/users/5c685740a045443997f807d9439f89c3/roles/478b9cd589e846588025a4b4ed2c07f0` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.449397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c380cc-b900-4d03-9e68-48dfd5ba6c0d None admin] RBAC: Authorizing `identity:create_grant(project_id=c0f0739222694cedabb8f12e4ce82e9b, user_id=5c685740a045443997f807d9439f89c3, role_id=478b9cd589e846588025a4b4ed2c07f0)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:53.450916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c380cc-b900-4d03-9e68-48dfd5ba6c0d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:53.465297 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 64/132] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:33:53 2022] PUT /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b/users/5c685740a045443997f807d9439f89c3/roles/478b9cd589e846588025a4b4ed2c07f0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:53.466329 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c380cc-b900-4d03-9e68-48dfd5ba6c0d None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:53.566466 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897574b0-2085-4b7d-aa6d-82d807c05e44 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:53.577344 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c380cc-b900-4d03-9e68-48dfd5ba6c0d None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:53.608657 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72328fef-a9ac-4a7d-a01f-b3eb9103d5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:53.610217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72328fef-a9ac-4a7d-a01f-b3eb9103d5b2 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.610735 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72328fef-a9ac-4a7d-a01f-b3eb9103d5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.611238 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72328fef-a9ac-4a7d-a01f-b3eb9103d5b2 None admin] PATH_INFO: `/v3/projects/1b1e8d351c924a2ab852be2c298ce115` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.613145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72328fef-a9ac-4a7d-a01f-b3eb9103d5b2 None admin] RBAC: Authorizing `identity:get_project(project_id=1b1e8d351c924a2ab852be2c298ce115)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:53.614970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d4a78-5d2e-4f08-987a-507f0cf875fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:53.615547 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72328fef-a9ac-4a7d-a01f-b3eb9103d5b2 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:53.615875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d4a78-5d2e-4f08-987a-507f0cf875fb None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.616148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d4a78-5d2e-4f08-987a-507f0cf875fb None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.616427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d4a78-5d2e-4f08-987a-507f0cf875fb None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.617736 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d4a78-5d2e-4f08-987a-507f0cf875fb None admin] RBAC: Authorizing `identity:get_project(project_id=c0f0739222694cedabb8f12e4ce82e9b)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:53.618995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 69/133] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:33:53 2022] GET /identity/v3/projects/1b1e8d351c924a2ab852be2c298ce115 => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:53.620352 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d4a78-5d2e-4f08-987a-507f0cf875fb None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:53.621091 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72328fef-a9ac-4a7d-a01f-b3eb9103d5b2 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:53.622923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 65/134] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:33:53 2022] GET /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:53.626017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d4a78-5d2e-4f08-987a-507f0cf875fb None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:53.635806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:53.661867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:53.749780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d4a78-5d2e-4f08-987a-507f0cf875fb None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:53.773195 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72328fef-a9ac-4a7d-a01f-b3eb9103d5b2 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:53.792429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78bdfc-1090-4120-aeeb-5a528d18a6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:53.793619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78bdfc-1090-4120-aeeb-5a528d18a6f5 None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.794407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78bdfc-1090-4120-aeeb-5a528d18a6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.794702 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78bdfc-1090-4120-aeeb-5a528d18a6f5 None admin] PATH_INFO: `/v3/projects/1b1e8d351c924a2ab852be2c298ce115/users/5c685740a045443997f807d9439f89c3/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.796839 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78bdfc-1090-4120-aeeb-5a528d18a6f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=1b1e8d351c924a2ab852be2c298ce115, user_id=5c685740a045443997f807d9439f89c3, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:53.800707 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78bdfc-1090-4120-aeeb-5a528d18a6f5 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:53.820243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 66/135] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:33:53 2022] PUT /identity/v3/projects/1b1e8d351c924a2ab852be2c298ce115/users/5c685740a045443997f807d9439f89c3/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:53.824391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78bdfc-1090-4120-aeeb-5a528d18a6f5 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:53.824579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e970a-b586-4c23-8557-339bf378780f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:53.824579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e970a-b586-4c23-8557-339bf378780f None admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.824579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e970a-b586-4c23-8557-339bf378780f None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.824579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e970a-b586-4c23-8557-339bf378780f None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b/users/4b433ad0233c4e5dbe8fad9896962e25/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.827481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e970a-b586-4c23-8557-339bf378780f None admin] RBAC: Authorizing `identity:create_grant(project_id=c0f0739222694cedabb8f12e4ce82e9b, user_id=4b433ad0233c4e5dbe8fad9896962e25, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:53.828814 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e970a-b586-4c23-8557-339bf378780f None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:53.849319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 70/136] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:33:53 2022] PUT /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b/users/4b433ad0233c4e5dbe8fad9896962e25/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:53.853307 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa3af1-b770-459c-9cf5-d803cbceec0d None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.853793 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa3af1-b770-459c-9cf5-d803cbceec0d None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.854217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa3af1-b770-459c-9cf5-d803cbceec0d None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.855743 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 71/137] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:53.865070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4f446-2927-4b08-8651-0357d8fc82b5 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:53.865376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4f446-2927-4b08-8651-0357d8fc82b5 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:53.865657 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4f446-2927-4b08-8651-0357d8fc82b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:53.909321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-3ee4f446-2927-4b08-8651-0357d8fc82b5 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:53.910691 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-3ee4f446-2927-4b08-8651-0357d8fc82b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:53.972603 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:54.021176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78bdfc-1090-4120-aeeb-5a528d18a6f5 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:54.043770 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 72/138] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:54.070618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2d115-37e7-4c7b-9d51-e07441e847e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:54.073875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2d115-37e7-4c7b-9d51-e07441e847e1 None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:54.074245 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2d115-37e7-4c7b-9d51-e07441e847e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:54.074551 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2d115-37e7-4c7b-9d51-e07441e847e1 None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b/users/5c685740a045443997f807d9439f89c3/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:54.075717 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4f446-2927-4b08-8651-0357d8fc82b5 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:54.077924 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2d115-37e7-4c7b-9d51-e07441e847e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0f0739222694cedabb8f12e4ce82e9b, user_id=5c685740a045443997f807d9439f89c3, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:54.079446 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2d115-37e7-4c7b-9d51-e07441e847e1 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:54.116743 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 67/139] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:33:53 2022] PUT /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b/users/5c685740a045443997f807d9439f89c3/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:54.122710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:54.227273 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4f446-2927-4b08-8651-0357d8fc82b5 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:54.268097 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff09cb6-3bd9-4a23-a88a-dd76a8408407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:54.269332 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff09cb6-3bd9-4a23-a88a-dd76a8408407 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:54.269615 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff09cb6-3bd9-4a23-a88a-dd76a8408407 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:54.269923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff09cb6-3bd9-4a23-a88a-dd76a8408407 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:54.271186 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff09cb6-3bd9-4a23-a88a-dd76a8408407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:54.272791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff09cb6-3bd9-4a23-a88a-dd76a8408407 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:54.275257 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 73/140] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:33:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:54.280971 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2d115-37e7-4c7b-9d51-e07441e847e1 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:54.428753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2d115-37e7-4c7b-9d51-e07441e847e1 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:54.444068 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082dc110-786f-4ef6-aedf-4589adc3b0cb None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:54.444374 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082dc110-786f-4ef6-aedf-4589adc3b0cb None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:54.444656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082dc110-786f-4ef6-aedf-4589adc3b0cb None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:54.446482 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 74/141] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:54.456352 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:54.468274 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf365e-670a-4294-97f7-231fd9e8beee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:54.468962 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf365e-670a-4294-97f7-231fd9e8beee None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:54.469254 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf365e-670a-4294-97f7-231fd9e8beee None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:54.469355 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d698a54-06d5-4f0d-b995-daddc62f9b70 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:54.469528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf365e-670a-4294-97f7-231fd9e8beee None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:54.470018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d698a54-06d5-4f0d-b995-daddc62f9b70 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:54.470294 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf365e-670a-4294-97f7-231fd9e8beee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:54.470695 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d698a54-06d5-4f0d-b995-daddc62f9b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:54.471816 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf365e-670a-4294-97f7-231fd9e8beee None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:54.502519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 68/142] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jun 27 15:33:54 2022] POST /identity/v3/users => generated 283 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:33:54.507012 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-7d698a54-06d5-4f0d-b995-daddc62f9b70 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:54.508314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-7d698a54-06d5-4f0d-b995-daddc62f9b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:54.593290 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 75/143] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:54.618457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf365e-670a-4294-97f7-231fd9e8beee None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:54.749346 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf365e-670a-4294-97f7-231fd9e8beee None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:54.807335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dbf893-3c14-425f-8f2a-b7bcd222818f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:54.808110 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbf893-3c14-425f-8f2a-b7bcd222818f None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:54.808403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbf893-3c14-425f-8f2a-b7bcd222818f None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:54.808684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbf893-3c14-425f-8f2a-b7bcd222818f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:54.823921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dbf893-3c14-425f-8f2a-b7bcd222818f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:33:54.840313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-84dbf893-3c14-425f-8f2a-b7bcd222818f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:33:54.840313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:54.840313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:54.840313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:54.840313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:54.840313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:54.840313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:54.840313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:54.840313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:54.840313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:54.840313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:54.840313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:54.840313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:33:54.840313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:33:54.840313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:33:54.840313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:33:54.841234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:33:54.841234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:33:54.841234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:33:54.841234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:33:54.841234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:54.841234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:54.841234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:54.841234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:54.841234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:54.841234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:54.841234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:54.841234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:54.841234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:54.841234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:54.841234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:54.841234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:54.842090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:54.842090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:54.842090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:54.842090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:54.842090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:33:54.842090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:33:54.842090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:33:54.842090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:33:54.842090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:33:54.842090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:33:54.842090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:33:54.842090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:33:54.842090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 69/144] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:33:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:33:54.849383 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d698a54-06d5-4f0d-b995-daddc62f9b70 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:54.874201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:54.986294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d698a54-06d5-4f0d-b995-daddc62f9b70 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:55.038715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0592f7-4846-480c-bd2f-f4afc31b2c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:55.039414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0592f7-4846-480c-bd2f-f4afc31b2c7d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:55.039703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0592f7-4846-480c-bd2f-f4afc31b2c7d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:55.040018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0592f7-4846-480c-bd2f-f4afc31b2c7d None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:55.041278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0592f7-4846-480c-bd2f-f4afc31b2c7d None admin] RBAC: Adding query filter params (name=reader) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:55.041548 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0592f7-4846-480c-bd2f-f4afc31b2c7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:55.043044 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0592f7-4846-480c-bd2f-f4afc31b2c7d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:55.052347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 76/145] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:33:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:55.058854 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dbf893-3c14-425f-8f2a-b7bcd222818f None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:55.126528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:55.210631 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dbf893-3c14-425f-8f2a-b7bcd222818f None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:55.256418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ec183-0aa2-45ee-bff5-a193eae91f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:55.257303 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ec183-0aa2-45ee-bff5-a193eae91f34 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:55.257591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ec183-0aa2-45ee-bff5-a193eae91f34 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:55.257905 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ec183-0aa2-45ee-bff5-a193eae91f34 None admin] PATH_INFO: `/v3/users/67655b82ae8f46caaa6337fa70edd15b` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:55.260541 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ec183-0aa2-45ee-bff5-a193eae91f34 None admin] RBAC: Authorizing `identity:get_user(user_id=67655b82ae8f46caaa6337fa70edd15b)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:55.262058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ec183-0aa2-45ee-bff5-a193eae91f34 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:55.263911 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 70/146] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:33:55 2022] GET /identity/v3/users/67655b82ae8f46caaa6337fa70edd15b => generated 289 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:55.269205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0592f7-4846-480c-bd2f-f4afc31b2c7d None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:55.366492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:55.393574 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0592f7-4846-480c-bd2f-f4afc31b2c7d None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:55.438332 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f2610-0af1-4454-b187-79df50347bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:55.439560 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f2610-0af1-4454-b187-79df50347bb3 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:55.440049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f2610-0af1-4454-b187-79df50347bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:55.440486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f2610-0af1-4454-b187-79df50347bb3 None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:55.441908 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f2610-0af1-4454-b187-79df50347bb3 None admin] RBAC: Authorizing `identity:get_project(project_id=c0f0739222694cedabb8f12e4ce82e9b)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:55.443516 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f2610-0af1-4454-b187-79df50347bb3 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:55.445699 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 77/147] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:33:55 2022] GET /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:55.451567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ec183-0aa2-45ee-bff5-a193eae91f34 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:55.567825 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ec183-0aa2-45ee-bff5-a193eae91f34 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:55.579220 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:55.611836 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafcf504-3c10-4bb5-86e7-c059f4451d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:55.612779 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcf504-3c10-4bb5-86e7-c059f4451d5e None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:55.613069 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcf504-3c10-4bb5-86e7-c059f4451d5e None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:55.613348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcf504-3c10-4bb5-86e7-c059f4451d5e None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b/users/67655b82ae8f46caaa6337fa70edd15b/roles/7a691f6f838a4a25a074d1537bb677a6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:55.615417 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafcf504-3c10-4bb5-86e7-c059f4451d5e None admin] RBAC: Authorizing `identity:create_grant(project_id=c0f0739222694cedabb8f12e4ce82e9b, user_id=67655b82ae8f46caaa6337fa70edd15b, role_id=7a691f6f838a4a25a074d1537bb677a6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:55.616947 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafcf504-3c10-4bb5-86e7-c059f4451d5e None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:55.638142 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 71/148] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:33:55 2022] PUT /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b/users/67655b82ae8f46caaa6337fa70edd15b/roles/7a691f6f838a4a25a074d1537bb677a6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:33:56.538049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21742bb-da9f-4320-981d-2a9561389a11 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:56.538588 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21742bb-da9f-4320-981d-2a9561389a11 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:56.538892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21742bb-da9f-4320-981d-2a9561389a11 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:56.541202 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 78/149] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:56.547010 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8163fe-3705-4c1e-a983-66855ba9552e None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:56.547495 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8163fe-3705-4c1e-a983-66855ba9552e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:56.547762 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8163fe-3705-4c1e-a983-66855ba9552e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:56.566088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-fd8163fe-3705-4c1e-a983-66855ba9552e None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:56.567469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-fd8163fe-3705-4c1e-a983-66855ba9552e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:56.658307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 72/150] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:33:56.682309 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21742bb-da9f-4320-981d-2a9561389a11 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:56.812542 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21742bb-da9f-4320-981d-2a9561389a11 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:56.860411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f3cd1-ccc3-44be-8f76-e074e2d661b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:56.861287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f3cd1-ccc3-44be-8f76-e074e2d661b4 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:56.861571 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f3cd1-ccc3-44be-8f76-e074e2d661b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:56.861869 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f3cd1-ccc3-44be-8f76-e074e2d661b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:56.876216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f3cd1-ccc3-44be-8f76-e074e2d661b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:33:56.885663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-743f3cd1-ccc3-44be-8f76-e074e2d661b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:33:56.885663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:56.885663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:56.885663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:56.885663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:56.885663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:56.885663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:56.885663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:56.885663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:56.885663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:56.885663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:56.885663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:56.885663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:33:56.885663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:33:56.885663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:33:56.885663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:33:56.886546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:33:56.886546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:33:56.886546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:33:56.886546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:33:56.886546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:56.886546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:56.886546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:56.886546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:56.886546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:56.886546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:56.886546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:56.886546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:56.886546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:56.886546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:56.886546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:56.886546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:56.887740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:56.887740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:56.887740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:56.887740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:56.887740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:33:56.887740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:33:56.887740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:33:56.887740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:33:56.887740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:33:56.887740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:33:56.887740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:33:56.887740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:33:56.887740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 79/151] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:33:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:33:56.893162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8163fe-3705-4c1e-a983-66855ba9552e None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:57.034394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:57.039025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8163fe-3705-4c1e-a983-66855ba9552e None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:57.090359 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600a0be-94ca-46f8-a969-60265249bba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:57.091180 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600a0be-94ca-46f8-a969-60265249bba6 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:57.091591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600a0be-94ca-46f8-a969-60265249bba6 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:57.091874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600a0be-94ca-46f8-a969-60265249bba6 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:57.092680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600a0be-94ca-46f8-a969-60265249bba6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:57.092990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600a0be-94ca-46f8-a969-60265249bba6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:57.094425 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600a0be-94ca-46f8-a969-60265249bba6 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:57.104213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 73/152] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:33:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:57.109358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f3cd1-ccc3-44be-8f76-e074e2d661b4 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:57.125319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:57.264425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f3cd1-ccc3-44be-8f76-e074e2d661b4 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:57.314924 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e119af-f175-49c8-b0b2-62c3a7936803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:57.315813 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e119af-f175-49c8-b0b2-62c3a7936803 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:57.316095 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e119af-f175-49c8-b0b2-62c3a7936803 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:57.316369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e119af-f175-49c8-b0b2-62c3a7936803 None admin] PATH_INFO: `/v3/users/5c685740a045443997f807d9439f89c3` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:57.317607 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e119af-f175-49c8-b0b2-62c3a7936803 None admin] RBAC: Authorizing `identity:get_user(user_id=5c685740a045443997f807d9439f89c3)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:57.319235 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e119af-f175-49c8-b0b2-62c3a7936803 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:57.320892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 80/153] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:33:57 2022] GET /identity/v3/users/5c685740a045443997f807d9439f89c3 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:57.326607 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600a0be-94ca-46f8-a969-60265249bba6 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:57.444329 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600a0be-94ca-46f8-a969-60265249bba6 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:57.485856 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123c629-5f6e-47c8-90e9-6770cee6e01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:57.486567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123c629-5f6e-47c8-90e9-6770cee6e01b None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:57.486851 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123c629-5f6e-47c8-90e9-6770cee6e01b None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:57.487125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123c629-5f6e-47c8-90e9-6770cee6e01b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:57.521438 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:57.555424 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4c750-fbf5-4661-a77e-74c78986dd3a None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:57.556503 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4c750-fbf5-4661-a77e-74c78986dd3a None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:57.556574 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4c750-fbf5-4661-a77e-74c78986dd3a None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:57.558635 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 81/154] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:57.565535 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890cd26-48bf-42ec-9e1a-58745940a03f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:57.565847 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890cd26-48bf-42ec-9e1a-58745940a03f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:57.566129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890cd26-48bf-42ec-9e1a-58745940a03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:57.585214 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-d890cd26-48bf-42ec-9e1a-58745940a03f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:57.591194 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123c629-5f6e-47c8-90e9-6770cee6e01b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:57.592682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123c629-5f6e-47c8-90e9-6770cee6e01b None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:57.594620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-d890cd26-48bf-42ec-9e1a-58745940a03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:57.708034 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 82/155] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:57.759934 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890cd26-48bf-42ec-9e1a-58745940a03f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:57.761900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 74/156] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:33:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:33:57.763810 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:57.767417 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123c629-5f6e-47c8-90e9-6770cee6e01b None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:57.880080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:57.951704 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123c629-5f6e-47c8-90e9-6770cee6e01b None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:57.994869 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad25ecc-6b5d-4063-95f3-7cb2e62d50c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:57.995966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad25ecc-6b5d-4063-95f3-7cb2e62d50c5 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:57.996248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad25ecc-6b5d-4063-95f3-7cb2e62d50c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:57.996523 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad25ecc-6b5d-4063-95f3-7cb2e62d50c5 None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:57.998028 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad25ecc-6b5d-4063-95f3-7cb2e62d50c5 None admin] RBAC: Authorizing `identity:get_project(project_id=c0f0739222694cedabb8f12e4ce82e9b)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:57.999497 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad25ecc-6b5d-4063-95f3-7cb2e62d50c5 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:58.001512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 75/157] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:33:57 2022] GET /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:58.003240 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4b57d-2fa0-4486-89f5-a0fd3c980dc5 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:58.003691 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4b57d-2fa0-4486-89f5-a0fd3c980dc5 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:58.004065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4b57d-2fa0-4486-89f5-a0fd3c980dc5 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:58.005572 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 76/158] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:58.011667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffe200-6b34-42eb-9a7b-7dc4ead712c7 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:58.012151 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffe200-6b34-42eb-9a7b-7dc4ead712c7 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:58.012436 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffe200-6b34-42eb-9a7b-7dc4ead712c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:58.031635 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890cd26-48bf-42ec-9e1a-58745940a03f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:58.035651 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-bdffe200-6b34-42eb-9a7b-7dc4ead712c7 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:58.037097 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-bdffe200-6b34-42eb-9a7b-7dc4ead712c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:58.119286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb2b84-f708-4a2e-89c3-7853694f0539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:58.120088 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb2b84-f708-4a2e-89c3-7853694f0539 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:58.120373 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb2b84-f708-4a2e-89c3-7853694f0539 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:58.120649 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb2b84-f708-4a2e-89c3-7853694f0539 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:58.151107 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb2b84-f708-4a2e-89c3-7853694f0539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:33:58.154979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 77/159] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:58.155896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdffe200-6b34-42eb-9a7b-7dc4ead712c7 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:58.159070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-73bb2b84-f708-4a2e-89c3-7853694f0539 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:33:58.159070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:58.159070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:58.159070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:58.159070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:58.159070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:58.159070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:58.159070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:58.159070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:58.159070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:58.159070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:58.159070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:58.159070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:33:58.159070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:33:58.159070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:33:58.159070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:33:58.159938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:33:58.159938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:33:58.159938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:33:58.159938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:33:58.159938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:58.159938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:58.159938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:58.159938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:58.159938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:58.159938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:58.159938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:58.159938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:58.159938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:58.159938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:58.159938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:58.159938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:58.161118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:58.161118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:58.161118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:58.161118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:58.161118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:33:58.161118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:33:58.161118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:33:58.161118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:33:58.161118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:33:58.161118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:33:58.161118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:33:58.161118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:33:58.161118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 83/160] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:33:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:33:58.170722 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb2b84-f708-4a2e-89c3-7853694f0539 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:58.281591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:58.317987 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:58.385663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdffe200-6b34-42eb-9a7b-7dc4ead712c7 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:58.390716 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb2b84-f708-4a2e-89c3-7853694f0539 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:58.434415 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655c5dc-3696-47c4-84f2-3bf4e5237558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:58.435370 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655c5dc-3696-47c4-84f2-3bf4e5237558 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:58.435499 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655c5dc-3696-47c4-84f2-3bf4e5237558 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:58.435706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655c5dc-3696-47c4-84f2-3bf4e5237558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:58.436564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655c5dc-3696-47c4-84f2-3bf4e5237558 None admin] RBAC: Adding query filter params (role.id=478b9cd589e846588025a4b4ed2c07f0, scope.project.id=c0f0739222694cedabb8f12e4ce82e9b, user.id=5c685740a045443997f807d9439f89c3) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:58.436878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655c5dc-3696-47c4-84f2-3bf4e5237558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:58.438414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655c5dc-3696-47c4-84f2-3bf4e5237558 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:58.452391 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda29db-cd46-4beb-aef7-a0f7e9b19888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:58.452589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 78/161] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:33:58 2022] GET /identity/v3/role_assignments?user.id=5c685740a045443997f807d9439f89c3&scope.project.id=c0f0739222694cedabb8f12e4ce82e9b&role.id=478b9cd589e846588025a4b4ed2c07f0 => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:58.455763 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af507a2-34d5-4917-ab27-4875c91e25b2 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:58.455763 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af507a2-34d5-4917-ab27-4875c91e25b2 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:58.455763 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af507a2-34d5-4917-ab27-4875c91e25b2 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:58.455763 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 79/162] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:58.456319 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af507a2-34d5-4917-ab27-4875c91e25b2 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:58.457137 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda29db-cd46-4beb-aef7-a0f7e9b19888 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:58.457621 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda29db-cd46-4beb-aef7-a0f7e9b19888 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:58.458116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda29db-cd46-4beb-aef7-a0f7e9b19888 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:58.459129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda29db-cd46-4beb-aef7-a0f7e9b19888 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:58.459724 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda29db-cd46-4beb-aef7-a0f7e9b19888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:58.461373 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda29db-cd46-4beb-aef7-a0f7e9b19888 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:58.487743 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 84/163] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:33:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:58.489075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82077f-7441-4a7d-931c-0fea8fbe8812 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:58.489367 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82077f-7441-4a7d-931c-0fea8fbe8812 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:58.489662 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82077f-7441-4a7d-931c-0fea8fbe8812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:58.531060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-be82077f-7441-4a7d-931c-0fea8fbe8812 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:58.532354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-be82077f-7441-4a7d-931c-0fea8fbe8812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:58.612109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af507a2-34d5-4917-ab27-4875c91e25b2 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:58.666073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:58.691577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da3de1-17dd-40bd-a5b9-fac6f8d7be71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:58.692394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da3de1-17dd-40bd-a5b9-fac6f8d7be71 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:58.692677 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da3de1-17dd-40bd-a5b9-fac6f8d7be71 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:58.692955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da3de1-17dd-40bd-a5b9-fac6f8d7be71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:58.706957 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 85/164] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:58.709006 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82077f-7441-4a7d-931c-0fea8fbe8812 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:58.714658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da3de1-17dd-40bd-a5b9-fac6f8d7be71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:33:58.723437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-21da3de1-17dd-40bd-a5b9-fac6f8d7be71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:33:58.723437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:58.723437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:58.723437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:58.723437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:58.723437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:58.723437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:58.723437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:58.723437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:58.723437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:58.723437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:58.723437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:58.723437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:33:58.723437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:33:58.723437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:33:58.723437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:33:58.724320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:33:58.724320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:33:58.724320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:33:58.724320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:33:58.724320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:58.724320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:58.724320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:58.724320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:58.724320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:58.724320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:58.724320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:58.724320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:58.724320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:58.724320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:58.724320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:58.724320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:58.725092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:58.725092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:58.725092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:58.725092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:58.725092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:33:58.725092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:33:58.725092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:33:58.725092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:33:58.725092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:33:58.725092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:33:58.725092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:33:58.725092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:33:58.727657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 80/165] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:33:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:33:58.738730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da3de1-17dd-40bd-a5b9-fac6f8d7be71 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:58.780261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:58.829586 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82077f-7441-4a7d-931c-0fea8fbe8812 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:58.866609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60bd6b-16c0-4ae5-aefc-1f776eab4510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:58.867822 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60bd6b-16c0-4ae5-aefc-1f776eab4510 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:58.868800 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60bd6b-16c0-4ae5-aefc-1f776eab4510 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:58.869455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60bd6b-16c0-4ae5-aefc-1f776eab4510 None admin] PATH_INFO: `/v3/users/4b433ad0233c4e5dbe8fad9896962e25` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:58.871084 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60bd6b-16c0-4ae5-aefc-1f776eab4510 None admin] RBAC: Authorizing `identity:get_user(user_id=4b433ad0233c4e5dbe8fad9896962e25)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:58.872737 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60bd6b-16c0-4ae5-aefc-1f776eab4510 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:58.874686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 86/166] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:33:58 2022] GET /identity/v3/users/4b433ad0233c4e5dbe8fad9896962e25 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:58.875630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60bd6b-16c0-4ae5-aefc-1f776eab4510 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:58.951556 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da3de1-17dd-40bd-a5b9-fac6f8d7be71 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:58.985422 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60bd6b-16c0-4ae5-aefc-1f776eab4510 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:59.012419 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:59.019471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0151d-1ca6-4335-994b-0d9a3c9c52a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:59.020456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0151d-1ca6-4335-994b-0d9a3c9c52a5 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:59.020992 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0151d-1ca6-4335-994b-0d9a3c9c52a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:59.021513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0151d-1ca6-4335-994b-0d9a3c9c52a5 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:59.022735 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0151d-1ca6-4335-994b-0d9a3c9c52a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:59.023308 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0151d-1ca6-4335-994b-0d9a3c9c52a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:59.023939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17527ea-bd90-4ea7-b85c-4a86831456af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:59.025959 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17527ea-bd90-4ea7-b85c-4a86831456af None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:59.026459 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17527ea-bd90-4ea7-b85c-4a86831456af None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:59.026912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17527ea-bd90-4ea7-b85c-4a86831456af None admin] PATH_INFO: `/v3/roles/member` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:59.029732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0151d-1ca6-4335-994b-0d9a3c9c52a5 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:59.047543 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 81/167] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:33:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:59.050893 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0151d-1ca6-4335-994b-0d9a3c9c52a5 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:59.052715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17527ea-bd90-4ea7-b85c-4a86831456af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:33:59.064349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-b17527ea-bd90-4ea7-b85c-4a86831456af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:33:59.064349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:33:59.064349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:33:59.064349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:33:59.064349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:33:59.064349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:33:59.064349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:33:59.064349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:33:59.064349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:33:59.064349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:33:59.064349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:33:59.064349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:33:59.064349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:33:59.064349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:33:59.064349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:33:59.064349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:33:59.065284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:33:59.065284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:33:59.065284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:33:59.065284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:33:59.065284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:33:59.065284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:33:59.065284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:33:59.065284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:33:59.065284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:33:59.065284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:33:59.065284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:33:59.065284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:33:59.065284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:33:59.065284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:33:59.065284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:33:59.065284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:33:59.066167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:33:59.066167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:33:59.066167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:33:59.066167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:33:59.066167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:33:59.066167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:33:59.066167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:33:59.066167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:33:59.066167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:33:59.066167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:33:59.066167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:33:59.066167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:33:59.071895 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 87/168] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:33:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:33:59.072753 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17527ea-bd90-4ea7-b85c-4a86831456af None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:59.162290 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:59.286046 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0151d-1ca6-4335-994b-0d9a3c9c52a5 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:59.359845 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17527ea-bd90-4ea7-b85c-4a86831456af None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:59.360741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f8547-c77a-4ef8-8645-55d1d8cf09dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:59.361792 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f8547-c77a-4ef8-8645-55d1d8cf09dd None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:59.362222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f8547-c77a-4ef8-8645-55d1d8cf09dd None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:59.362641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f8547-c77a-4ef8-8645-55d1d8cf09dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:59.448829 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c349a19-a645-49fe-b373-42e38fe82377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:59.449988 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c349a19-a645-49fe-b373-42e38fe82377 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:59.450417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c349a19-a645-49fe-b373-42e38fe82377 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:59.450837 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c349a19-a645-49fe-b373-42e38fe82377 None admin] PATH_INFO: `/v3/users/5c685740a045443997f807d9439f89c3` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:59.454263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c349a19-a645-49fe-b373-42e38fe82377 None admin] RBAC: Authorizing `identity:get_user(user_id=5c685740a045443997f807d9439f89c3)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:59.455945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c349a19-a645-49fe-b373-42e38fe82377 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:59.457808 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 88/169] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:33:59 2022] GET /identity/v3/users/5c685740a045443997f807d9439f89c3 => generated 275 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:59.460024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c349a19-a645-49fe-b373-42e38fe82377 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:59.533628 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:59.542069 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f8547-c77a-4ef8-8645-55d1d8cf09dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:59.543556 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f8547-c77a-4ef8-8645-55d1d8cf09dd None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:59.615647 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:59.630049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c349a19-a645-49fe-b373-42e38fe82377 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:59.674908 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262da6cb-4e5d-4066-8afc-64fd3dfac22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:59.675819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262da6cb-4e5d-4066-8afc-64fd3dfac22b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:59.676245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262da6cb-4e5d-4066-8afc-64fd3dfac22b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:59.676374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 82/170] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:33:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:33:59.676758 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262da6cb-4e5d-4066-8afc-64fd3dfac22b None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:59.677672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262da6cb-4e5d-4066-8afc-64fd3dfac22b None admin] RBAC: Adding query filter params (name=member) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:33:59.677815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f75290-6c6b-44ac-9d47-86df347f0bfa None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:59.678269 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262da6cb-4e5d-4066-8afc-64fd3dfac22b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:59.678493 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f75290-6c6b-44ac-9d47-86df347f0bfa None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:59.678899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f75290-6c6b-44ac-9d47-86df347f0bfa None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:59.679931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262da6cb-4e5d-4066-8afc-64fd3dfac22b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:59.680574 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 83/171] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:59.682556 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f75290-6c6b-44ac-9d47-86df347f0bfa None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:59.695008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 89/172] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:33:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:59.697062 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f12e2-57b1-4604-b727-fa5e9e6b7905 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:59.697601 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f12e2-57b1-4604-b727-fa5e9e6b7905 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:59.698606 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f12e2-57b1-4604-b727-fa5e9e6b7905 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:59.700486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 90/173] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:59.701794 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a3c86-92d2-4a8c-b6b4-fb2381f08f01 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:59.702243 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a3c86-92d2-4a8c-b6b4-fb2381f08f01 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:59.702892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a3c86-92d2-4a8c-b6b4-fb2381f08f01 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:59.704290 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 91/174] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:33:59.705123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a3c86-92d2-4a8c-b6b4-fb2381f08f01 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:33:59.803187 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f75290-6c6b-44ac-9d47-86df347f0bfa None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:59.819830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a3c86-92d2-4a8c-b6b4-fb2381f08f01 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:33:59.845179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2c134-1d7c-40c8-b4c7-cabf9fbb7d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:59.845915 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2c134-1d7c-40c8-b4c7-cabf9fbb7d71 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:59.846282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2c134-1d7c-40c8-b4c7-cabf9fbb7d71 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:59.846560 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2c134-1d7c-40c8-b4c7-cabf9fbb7d71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:59.854364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:59.865315 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:33:59.879027 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a55c20-d17b-4f8f-9e2d-55dda6460ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:33:59.879893 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a55c20-d17b-4f8f-9e2d-55dda6460ec3 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:59.880173 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a55c20-d17b-4f8f-9e2d-55dda6460ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:59.880446 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a55c20-d17b-4f8f-9e2d-55dda6460ec3 None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:59.881704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a55c20-d17b-4f8f-9e2d-55dda6460ec3 None admin] RBAC: Authorizing `identity:get_project(project_id=c0f0739222694cedabb8f12e4ce82e9b)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:59.883162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a55c20-d17b-4f8f-9e2d-55dda6460ec3 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:59.885594 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 92/175] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:33:59 2022] GET /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:33:59.886679 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37031a9a-610b-42e3-be41-56e76b49e513 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:33:59.886971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37031a9a-610b-42e3-be41-56e76b49e513 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:33:59.887311 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37031a9a-610b-42e3-be41-56e76b49e513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:33:59.904516 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-37031a9a-610b-42e3-be41-56e76b49e513 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:33:59.905757 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-37031a9a-610b-42e3-be41-56e76b49e513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:33:59.930161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2c134-1d7c-40c8-b4c7-cabf9fbb7d71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:33:59.931714 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb2c134-1d7c-40c8-b4c7-cabf9fbb7d71 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:33:59.981749 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 93/176] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:33:59.981749 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37031a9a-610b-42e3-be41-56e76b49e513 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:00.062444 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 84/177] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:33:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:00.063628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb3ac0-f2e2-4d68-9b7a-c3f5c76a1f9c None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:00.063928 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb3ac0-f2e2-4d68-9b7a-c3f5c76a1f9c None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:00.064207 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb3ac0-f2e2-4d68-9b7a-c3f5c76a1f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:00.081250 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-1ceb3ac0-f2e2-4d68-9b7a-c3f5c76a1f9c None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:00.082445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1ceb3ac0-f2e2-4d68-9b7a-c3f5c76a1f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:00.103056 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37031a9a-610b-42e3-be41-56e76b49e513 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:00.119334 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:00.131446 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:00.142911 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7324b39b-a29e-4fe9-aa2d-db917e76e090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:00.143792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324b39b-a29e-4fe9-aa2d-db917e76e090 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:00.144082 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324b39b-a29e-4fe9-aa2d-db917e76e090 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:00.144353 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324b39b-a29e-4fe9-aa2d-db917e76e090 None admin] PATH_INFO: `/v3/users/5c685740a045443997f807d9439f89c3` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:00.145555 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7324b39b-a29e-4fe9-aa2d-db917e76e090 None admin] RBAC: Authorizing `identity:get_user(user_id=5c685740a045443997f807d9439f89c3)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:00.147019 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7324b39b-a29e-4fe9-aa2d-db917e76e090 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:00.153519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 94/178] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:33:59 2022] GET /identity/v3/users/5c685740a045443997f807d9439f89c3 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:00.153519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395c897-e1a2-421c-930d-622801799da6 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:00.153519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395c897-e1a2-421c-930d-622801799da6 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:00.153519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395c897-e1a2-421c-930d-622801799da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:00.157732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 85/179] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:00.158559 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb3ac0-f2e2-4d68-9b7a-c3f5c76a1f9c None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:00.173331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-b395c897-e1a2-421c-930d-622801799da6 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:00.174535 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-b395c897-e1a2-421c-930d-622801799da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:00.249197 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 95/180] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:00.250596 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395c897-e1a2-421c-930d-622801799da6 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:00.264374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb3ac0-f2e2-4d68-9b7a-c3f5c76a1f9c None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:00.300504 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90d22f-eef2-4479-8ade-f5abc9843c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:00.301517 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90d22f-eef2-4479-8ade-f5abc9843c35 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:00.301849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90d22f-eef2-4479-8ade-f5abc9843c35 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:00.302125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90d22f-eef2-4479-8ade-f5abc9843c35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:00.302972 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90d22f-eef2-4479-8ade-f5abc9843c35 None admin] RBAC: Adding query filter params (role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.project.id=c0f0739222694cedabb8f12e4ce82e9b, user.id=4b433ad0233c4e5dbe8fad9896962e25) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:00.303289 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90d22f-eef2-4479-8ade-f5abc9843c35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:00.304740 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90d22f-eef2-4479-8ade-f5abc9843c35 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:00.311358 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 86/181] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:34:00 2022] GET /identity/v3/role_assignments?user.id=4b433ad0233c4e5dbe8fad9896962e25&scope.project.id=c0f0739222694cedabb8f12e4ce82e9b&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:00.318780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90d22f-eef2-4479-8ade-f5abc9843c35 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:00.364498 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395c897-e1a2-421c-930d-622801799da6 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:00.371437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:00.379611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:00.409227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81d7d5-acf0-421f-a1b3-2305965fc48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:00.410087 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81d7d5-acf0-421f-a1b3-2305965fc48d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:00.410374 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81d7d5-acf0-421f-a1b3-2305965fc48d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:00.410650 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81d7d5-acf0-421f-a1b3-2305965fc48d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:00.424478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81d7d5-acf0-421f-a1b3-2305965fc48d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:00.433080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-af81d7d5-acf0-421f-a1b3-2305965fc48d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:34:00.433080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:00.433080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:00.433080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:00.433080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:00.433080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:00.433080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:00.433080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:00.433080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:00.433080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:00.433080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:00.433080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:00.433080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:00.433080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:00.433080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:00.433080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:00.434365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:00.434365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:00.434365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:00.434365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:00.434365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:00.434365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:00.434365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:00.434365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:00.434365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:00.434365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:00.434365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:00.434365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:00.434365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:00.434365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:00.434365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:00.434365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:00.435174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:00.435174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:00.435174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:00.435174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:00.435174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:00.435174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:00.435174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:00.435174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:00.435174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:00.435174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:00.435174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:34:00.435174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:00.435174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 96/182] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:34:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:00.435913 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5962d8d-b1c4-4f01-bb18-ff40961548db None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:00.436187 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5962d8d-b1c4-4f01-bb18-ff40961548db None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:00.436348 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5962d8d-b1c4-4f01-bb18-ff40961548db None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:00.438241 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 97/183] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:00.442276 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5962d8d-b1c4-4f01-bb18-ff40961548db None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:00.449991 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90d22f-eef2-4479-8ade-f5abc9843c35 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:00.490435 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfaf354-efcd-439a-93a5-d03b4c001d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:00.491469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfaf354-efcd-439a-93a5-d03b4c001d16 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:00.491757 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfaf354-efcd-439a-93a5-d03b4c001d16 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:00.492038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfaf354-efcd-439a-93a5-d03b4c001d16 None admin] PATH_INFO: `/v3/projects/1b1e8d351c924a2ab852be2c298ce115` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:00.493320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfaf354-efcd-439a-93a5-d03b4c001d16 None admin] RBAC: Authorizing `identity:get_project(project_id=1b1e8d351c924a2ab852be2c298ce115)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:00.494839 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfaf354-efcd-439a-93a5-d03b4c001d16 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:00.496597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 87/184] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:00 2022] GET /identity/v3/projects/1b1e8d351c924a2ab852be2c298ce115 => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:00.497462 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfaf354-efcd-439a-93a5-d03b4c001d16 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:00.563954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5962d8d-b1c4-4f01-bb18-ff40961548db None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:00.600832 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc5567-e842-4cd2-b94a-d7fb2d1fce45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:00.601483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc5567-e842-4cd2-b94a-d7fb2d1fce45 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:00.601812 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc5567-e842-4cd2-b94a-d7fb2d1fce45 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:00.602094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc5567-e842-4cd2-b94a-d7fb2d1fce45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:00.611531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfaf354-efcd-439a-93a5-d03b4c001d16 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:00.622163 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:00.642583 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:00.648186 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733458c-f83d-4657-94d8-7de9de5f99c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:00.649010 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733458c-f83d-4657-94d8-7de9de5f99c7 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:00.649303 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733458c-f83d-4657-94d8-7de9de5f99c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:00.649614 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733458c-f83d-4657-94d8-7de9de5f99c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:00.660730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1733458c-f83d-4657-94d8-7de9de5f99c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:00.669982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-1733458c-f83d-4657-94d8-7de9de5f99c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:00.669982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:00.669982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:00.669982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:00.669982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:00.669982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:00.669982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:00.669982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:00.669982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:00.669982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:00.669982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:00.669982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:00.669982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:00.669982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:00.669982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:00.669982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:00.670863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:00.670863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:00.670863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:00.670863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:00.670863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:00.670863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:00.670863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:00.670863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:00.670863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:00.670863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:00.670863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:00.670863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:00.670863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:00.670863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:00.670863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:00.670863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:00.671634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:00.671634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:00.671634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:00.671634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:00.671634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:00.671634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:00.671634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:00.671634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:00.671634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:00.671634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:00.671634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:00.671634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:00.671634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 88/185] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:34:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:00.672277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733458c-f83d-4657-94d8-7de9de5f99c7 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:00.673192 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc5567-e842-4cd2-b94a-d7fb2d1fce45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:00.674710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc5567-e842-4cd2-b94a-d7fb2d1fce45 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:00.778972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 98/186] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:00.779824 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733458c-f83d-4657-94d8-7de9de5f99c7 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:00.780044 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc5567-e842-4cd2-b94a-d7fb2d1fce45 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:00.817115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9b5aa-862d-499b-92f8-0d069ac8d45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:00.817878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9b5aa-862d-499b-92f8-0d069ac8d45a None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:00.818170 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9b5aa-862d-499b-92f8-0d069ac8d45a None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:00.818449 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9b5aa-862d-499b-92f8-0d069ac8d45a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:00.830727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a9b5aa-862d-499b-92f8-0d069ac8d45a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:00.838729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-08a9b5aa-862d-499b-92f8-0d069ac8d45a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:34:00.838729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:00.838729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:00.838729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:00.838729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:00.838729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:00.838729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:00.838729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:00.838729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:00.838729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:00.838729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:00.838729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:00.838729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:00.838729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:00.838729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:00.838729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:00.839682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:00.839682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:00.839682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:00.839682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:00.839682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:00.839682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:00.839682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:00.839682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:00.839682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:00.839682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:00.839682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:00.839682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:00.839682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:00.839682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:00.839682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:00.839682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:00.840466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:00.840466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:00.840466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:00.840466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:00.840466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:00.840466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:00.840466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:00.840466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:00.840466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:00.840466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:00.840466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:34:00.840466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:00.840466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 89/187] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:00.841968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f179c-2c75-46d2-9d7f-2c372a072ef6 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:00.841968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f179c-2c75-46d2-9d7f-2c372a072ef6 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:00.842124 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f179c-2c75-46d2-9d7f-2c372a072ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:00.854840 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:00.859779 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-4b9f179c-2c75-46d2-9d7f-2c372a072ef6 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:00.861001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4b9f179c-2c75-46d2-9d7f-2c372a072ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:00.885682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:00.887517 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc5567-e842-4cd2-b94a-d7fb2d1fce45 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:00.921771 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00eebf-ba50-4a03-9813-5c9a06681314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:00.922445 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00eebf-ba50-4a03-9813-5c9a06681314 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:00.922725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00eebf-ba50-4a03-9813-5c9a06681314 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:00.922994 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00eebf-ba50-4a03-9813-5c9a06681314 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:00.923742 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00eebf-ba50-4a03-9813-5c9a06681314 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:00.924047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00eebf-ba50-4a03-9813-5c9a06681314 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:00.925404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00eebf-ba50-4a03-9813-5c9a06681314 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:00.931709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 90/188] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:00.932564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f179c-2c75-46d2-9d7f-2c372a072ef6 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:00.942747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 99/189] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:00.942747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00eebf-ba50-4a03-9813-5c9a06681314 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:01.033235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f179c-2c75-46d2-9d7f-2c372a072ef6 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:01.044563 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00eebf-ba50-4a03-9813-5c9a06681314 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:01.068509 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8079f9-1c54-4268-9890-3fdb113f62b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:01.069087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8079f9-1c54-4268-9890-3fdb113f62b2 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:01.069363 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8079f9-1c54-4268-9890-3fdb113f62b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:01.069664 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8079f9-1c54-4268-9890-3fdb113f62b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:01.070515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8079f9-1c54-4268-9890-3fdb113f62b2 None admin] RBAC: Adding query filter params (role.id=5dc71eded4f64c08b2fb5f545b63b2f6, scope.project.id=1b1e8d351c924a2ab852be2c298ce115, user.id=5c685740a045443997f807d9439f89c3) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:01.070827 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8079f9-1c54-4268-9890-3fdb113f62b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:01.073414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8079f9-1c54-4268-9890-3fdb113f62b2 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:01.074080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:01.079571 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea44032-6145-4297-865d-403634f2660a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:01.079724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 91/190] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:34:00 2022] GET /identity/v3/role_assignments?user.id=5c685740a045443997f807d9439f89c3&scope.project.id=1b1e8d351c924a2ab852be2c298ce115&role.id=5dc71eded4f64c08b2fb5f545b63b2f6 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:01.090017 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea44032-6145-4297-865d-403634f2660a None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:01.090017 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea44032-6145-4297-865d-403634f2660a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:01.090222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8079f9-1c54-4268-9890-3fdb113f62b2 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:01.090297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea44032-6145-4297-865d-403634f2660a None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:01.090297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea44032-6145-4297-865d-403634f2660a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:01.090297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea44032-6145-4297-865d-403634f2660a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:01.090297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea44032-6145-4297-865d-403634f2660a None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:01.093547 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 100/191] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:01.094565 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea44032-6145-4297-865d-403634f2660a None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:01.125454 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:01.217657 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea44032-6145-4297-865d-403634f2660a None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:01.221480 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8079f9-1c54-4268-9890-3fdb113f62b2 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:01.258071 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717aadcb-2d5b-430c-9e05-baf079166e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:01.258796 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aadcb-2d5b-430c-9e05-baf079166e95 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:01.259125 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aadcb-2d5b-430c-9e05-baf079166e95 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:01.259405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aadcb-2d5b-430c-9e05-baf079166e95 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:01.260166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717aadcb-2d5b-430c-9e05-baf079166e95 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:01.260478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717aadcb-2d5b-430c-9e05-baf079166e95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:01.261903 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717aadcb-2d5b-430c-9e05-baf079166e95 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:01.262660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc8c5d-480f-4cd1-b66b-d5c30c8adc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:01.263696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc8c5d-480f-4cd1-b66b-d5c30c8adc35 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:01.263980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc8c5d-480f-4cd1-b66b-d5c30c8adc35 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:01.264259 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc8c5d-480f-4cd1-b66b-d5c30c8adc35 None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:01.265535 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc8c5d-480f-4cd1-b66b-d5c30c8adc35 None admin] RBAC: Authorizing `identity:get_project(project_id=c0f0739222694cedabb8f12e4ce82e9b)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:01.267068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc8c5d-480f-4cd1-b66b-d5c30c8adc35 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:01.268960 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 92/192] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:01 2022] GET /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:01.269678 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc8c5d-480f-4cd1-b66b-d5c30c8adc35 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:01.276959 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 101/193] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:01.278086 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717aadcb-2d5b-430c-9e05-baf079166e95 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:01.346439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:01.378321 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:01.389268 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc8c5d-480f-4cd1-b66b-d5c30c8adc35 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:01.390373 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717aadcb-2d5b-430c-9e05-baf079166e95 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:01.423741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618719bf-a2a0-46d4-8051-1a8485fe52b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:01.424411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c4bba-e29c-4398-9a06-e06ed0b854b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:01.424621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618719bf-a2a0-46d4-8051-1a8485fe52b3 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:01.424906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618719bf-a2a0-46d4-8051-1a8485fe52b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:01.425181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618719bf-a2a0-46d4-8051-1a8485fe52b3 None admin] PATH_INFO: `/v3/users/7f3816d6eade43a5a50b8d7b60acc26d` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:01.425277 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c4bba-e29c-4398-9a06-e06ed0b854b7 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:01.425480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c4bba-e29c-4398-9a06-e06ed0b854b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:01.425793 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c4bba-e29c-4398-9a06-e06ed0b854b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:01.426999 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c4bba-e29c-4398-9a06-e06ed0b854b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:01.428461 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c4bba-e29c-4398-9a06-e06ed0b854b7 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:01.430134 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 102/194] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:01.431461 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c4bba-e29c-4398-9a06-e06ed0b854b7 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:01.440566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618719bf-a2a0-46d4-8051-1a8485fe52b3 None admin] RBAC: Authorizing `identity:get_user(user_id=7f3816d6eade43a5a50b8d7b60acc26d)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:01.442080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618719bf-a2a0-46d4-8051-1a8485fe52b3 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:01.443813 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 93/195] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:01 2022] GET /identity/v3/users/7f3816d6eade43a5a50b8d7b60acc26d => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:01.444584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618719bf-a2a0-46d4-8051-1a8485fe52b3 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:01.535084 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c4bba-e29c-4398-9a06-e06ed0b854b7 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:01.546301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618719bf-a2a0-46d4-8051-1a8485fe52b3 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:01.560055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:01.571606 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688bac19-1371-4908-9261-2ab94d1c3a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:01.572717 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688bac19-1371-4908-9261-2ab94d1c3a35 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:01.573007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688bac19-1371-4908-9261-2ab94d1c3a35 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:01.573288 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688bac19-1371-4908-9261-2ab94d1c3a35 None admin] PATH_INFO: `/v3/users/7f3816d6eade43a5a50b8d7b60acc26d` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:01.574536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688bac19-1371-4908-9261-2ab94d1c3a35 None admin] RBAC: Authorizing `identity:get_user(user_id=7f3816d6eade43a5a50b8d7b60acc26d)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:01.575970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688bac19-1371-4908-9261-2ab94d1c3a35 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:01.577592 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 103/196] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:01 2022] GET /identity/v3/users/7f3816d6eade43a5a50b8d7b60acc26d => generated 283 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:01.578369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688bac19-1371-4908-9261-2ab94d1c3a35 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:01.582175 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f86c0-4d2b-4079-b82a-e83eccc5fec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:01.582924 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f86c0-4d2b-4079-b82a-e83eccc5fec7 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:01.583217 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f86c0-4d2b-4079-b82a-e83eccc5fec7 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:01.583499 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f86c0-4d2b-4079-b82a-e83eccc5fec7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:01.584353 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f86c0-4d2b-4079-b82a-e83eccc5fec7 None admin] RBAC: Adding query filter params (role.id=5dc71eded4f64c08b2fb5f545b63b2f6, scope.project.id=c0f0739222694cedabb8f12e4ce82e9b, user.id=5c685740a045443997f807d9439f89c3) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:01.584668 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f86c0-4d2b-4079-b82a-e83eccc5fec7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:01.586178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f86c0-4d2b-4079-b82a-e83eccc5fec7 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:01.592697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 94/197] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:34:01 2022] GET /identity/v3/role_assignments?user.id=5c685740a045443997f807d9439f89c3&scope.project.id=c0f0739222694cedabb8f12e4ce82e9b&role.id=5dc71eded4f64c08b2fb5f545b63b2f6 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:01.593477 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f86c0-4d2b-4079-b82a-e83eccc5fec7 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:01.623650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:01.704375 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688bac19-1371-4908-9261-2ab94d1c3a35 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:01.731930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f86c0-4d2b-4079-b82a-e83eccc5fec7 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:01.743799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1170c-7e3a-4a42-91ae-87144972a44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:01.744725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1170c-7e3a-4a42-91ae-87144972a44b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:01.745014 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1170c-7e3a-4a42-91ae-87144972a44b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:01.745291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1170c-7e3a-4a42-91ae-87144972a44b None admin] PATH_INFO: `/v3/users/67655b82ae8f46caaa6337fa70edd15b` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:01.746673 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1170c-7e3a-4a42-91ae-87144972a44b None admin] RBAC: Authorizing `identity:get_user(user_id=67655b82ae8f46caaa6337fa70edd15b)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:01.748177 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1170c-7e3a-4a42-91ae-87144972a44b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:01.749953 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 104/198] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:01 2022] GET /identity/v3/users/67655b82ae8f46caaa6337fa70edd15b => generated 289 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:01.750735 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1170c-7e3a-4a42-91ae-87144972a44b None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:01.778442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d35fbc-1e1d-4aef-842e-298841984150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:01.779135 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d35fbc-1e1d-4aef-842e-298841984150 None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:01.779427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d35fbc-1e1d-4aef-842e-298841984150 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:01.779712 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d35fbc-1e1d-4aef-842e-298841984150 None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:01.780437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d35fbc-1e1d-4aef-842e-298841984150 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:01.782022 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d35fbc-1e1d-4aef-842e-298841984150 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:01.808902 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 95/199] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jun 27 15:34:01 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:01.810360 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d35fbc-1e1d-4aef-842e-298841984150 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:01.830344 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:01.897113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1170c-7e3a-4a42-91ae-87144972a44b None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:01.904531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:01.941903 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5963096-6340-4dd5-8a0a-946d8ceab069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:01.942592 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5963096-6340-4dd5-8a0a-946d8ceab069 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:01.942865 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5963096-6340-4dd5-8a0a-946d8ceab069 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:01.943144 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5963096-6340-4dd5-8a0a-946d8ceab069 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:01.944411 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d35fbc-1e1d-4aef-842e-298841984150 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:01.987782 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfb49f-cc3d-478f-aae4-3d40f365a828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:01.988457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfb49f-cc3d-478f-aae4-3d40f365a828 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:01.988741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfb49f-cc3d-478f-aae4-3d40f365a828 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:01.989020 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfb49f-cc3d-478f-aae4-3d40f365a828 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:02.029084 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5963096-6340-4dd5-8a0a-946d8ceab069 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:02.030602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5963096-6340-4dd5-8a0a-946d8ceab069 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:02.081545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bfb49f-cc3d-478f-aae4-3d40f365a828 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:02.083079 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bfb49f-cc3d-478f-aae4-3d40f365a828 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:02.110246 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:02.159535 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 105/200] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:02.160162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5963096-6340-4dd5-8a0a-946d8ceab069 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:02.175446 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:02.218900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 96/201] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:02.219761 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfb49f-cc3d-478f-aae4-3d40f365a828 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:02.271090 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5963096-6340-4dd5-8a0a-946d8ceab069 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:02.316751 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229533c-694d-4ad4-8203-115a779d742f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:02.317418 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229533c-694d-4ad4-8203-115a779d742f None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:02.317799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229533c-694d-4ad4-8203-115a779d742f None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:02.318089 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229533c-694d-4ad4-8203-115a779d742f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:02.340886 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfb49f-cc3d-478f-aae4-3d40f365a828 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:02.349866 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:02.382616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48988b47-a67d-410f-81c9-0e0676780ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:02.383806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48988b47-a67d-410f-81c9-0e0676780ba4 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:02.384099 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48988b47-a67d-410f-81c9-0e0676780ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:02.384471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48988b47-a67d-410f-81c9-0e0676780ba4 None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:02.385951 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48988b47-a67d-410f-81c9-0e0676780ba4 None admin] RBAC: Authorizing `identity:get_project(project_id=9974edfe18e14b8a9711b3f5c02808a9)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:02.387608 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48988b47-a67d-410f-81c9-0e0676780ba4 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:02.389498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 97/202] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:02 2022] GET /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:02.390101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48988b47-a67d-410f-81c9-0e0676780ba4 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:02.394335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229533c-694d-4ad4-8203-115a779d742f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:02.395618 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229533c-694d-4ad4-8203-115a779d742f None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:02.428938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:02.507238 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 106/203] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:02.508171 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229533c-694d-4ad4-8203-115a779d742f None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:02.515353 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48988b47-a67d-410f-81c9-0e0676780ba4 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:02.565133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5b6ff-a854-4ece-9076-feac91266f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:02.566904 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5b6ff-a854-4ece-9076-feac91266f02 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:02.567534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5b6ff-a854-4ece-9076-feac91266f02 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:02.568119 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5b6ff-a854-4ece-9076-feac91266f02 None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:02.570310 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a5b6ff-a854-4ece-9076-feac91266f02 None admin] RBAC: Authorizing `identity:get_project(project_id=9974edfe18e14b8a9711b3f5c02808a9)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:02.572681 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a5b6ff-a854-4ece-9076-feac91266f02 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:02.573503 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:02.576854 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 98/204] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:02 2022] GET /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:02.578351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5b6ff-a854-4ece-9076-feac91266f02 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:02.620819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229533c-694d-4ad4-8203-115a779d742f None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:02.667153 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4581f0d-6981-48b9-b12d-75b4a005e0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:02.667824 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4581f0d-6981-48b9-b12d-75b4a005e0b5 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:02.668112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4581f0d-6981-48b9-b12d-75b4a005e0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:02.668390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4581f0d-6981-48b9-b12d-75b4a005e0b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:02.669237 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4581f0d-6981-48b9-b12d-75b4a005e0b5 None admin] RBAC: Adding query filter params (role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.project.id=9974edfe18e14b8a9711b3f5c02808a9, user.id=7f3816d6eade43a5a50b8d7b60acc26d) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:02.669552 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4581f0d-6981-48b9-b12d-75b4a005e0b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:02.672288 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4581f0d-6981-48b9-b12d-75b4a005e0b5 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:02.678553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 107/205] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:34:02 2022] GET /identity/v3/role_assignments?user.id=7f3816d6eade43a5a50b8d7b60acc26d&scope.project.id=9974edfe18e14b8a9711b3f5c02808a9&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:02.679255 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4581f0d-6981-48b9-b12d-75b4a005e0b5 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:02.733876 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5b6ff-a854-4ece-9076-feac91266f02 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:02.738557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:02.785804 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31fbbc5-2160-4a3f-b6f2-a0a82864b63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:02.787319 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fbbc5-2160-4a3f-b6f2-a0a82864b63d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:02.787807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fbbc5-2160-4a3f-b6f2-a0a82864b63d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:02.788305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fbbc5-2160-4a3f-b6f2-a0a82864b63d None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:02.789769 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31fbbc5-2160-4a3f-b6f2-a0a82864b63d None admin] RBAC: Authorizing `identity:get_project(project_id=c0f0739222694cedabb8f12e4ce82e9b)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:02.792061 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31fbbc5-2160-4a3f-b6f2-a0a82864b63d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:02.794285 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 99/206] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:02 2022] GET /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:02.801751 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31fbbc5-2160-4a3f-b6f2-a0a82864b63d None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:02.805301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4581f0d-6981-48b9-b12d-75b4a005e0b5 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:02.864777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:02.918400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9e6fd-2cf4-4046-9f97-486c8a29c5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:02.919382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9e6fd-2cf4-4046-9f97-486c8a29c5a3 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:02.919382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9e6fd-2cf4-4046-9f97-486c8a29c5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:02.919725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9e6fd-2cf4-4046-9f97-486c8a29c5a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:02.920436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9e6fd-2cf4-4046-9f97-486c8a29c5a3 None admin] RBAC: Adding query filter params (role.id=478b9cd589e846588025a4b4ed2c07f0, scope.project.id=9974edfe18e14b8a9711b3f5c02808a9, user.id=7f3816d6eade43a5a50b8d7b60acc26d) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:02.920755 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9e6fd-2cf4-4046-9f97-486c8a29c5a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:02.922246 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9e6fd-2cf4-4046-9f97-486c8a29c5a3 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:02.931684 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 108/207] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:34:02 2022] GET /identity/v3/role_assignments?user.id=7f3816d6eade43a5a50b8d7b60acc26d&scope.project.id=9974edfe18e14b8a9711b3f5c02808a9&role.id=478b9cd589e846588025a4b4ed2c07f0 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:02.954065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31fbbc5-2160-4a3f-b6f2-a0a82864b63d None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:02.992543 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb78366-0d92-42ba-bef2-7c852ac55e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:02.994127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb78366-0d92-42ba-bef2-7c852ac55e2d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:02.994560 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb78366-0d92-42ba-bef2-7c852ac55e2d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:02.994985 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb78366-0d92-42ba-bef2-7c852ac55e2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:02.996027 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb78366-0d92-42ba-bef2-7c852ac55e2d None admin] RBAC: Adding query filter params (role.id=7a691f6f838a4a25a074d1537bb677a6, scope.project.id=c0f0739222694cedabb8f12e4ce82e9b, user.id=67655b82ae8f46caaa6337fa70edd15b) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:02.996488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb78366-0d92-42ba-bef2-7c852ac55e2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:02.998138 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb78366-0d92-42ba-bef2-7c852ac55e2d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:03.005262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 100/208] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:34:02 2022] GET /identity/v3/role_assignments?user.id=67655b82ae8f46caaa6337fa70edd15b&scope.project.id=c0f0739222694cedabb8f12e4ce82e9b&role.id=7a691f6f838a4a25a074d1537bb677a6 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:04.207694 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f28c26-e4a9-4234-8ca4-f128d57ed10e None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:04.208699 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f28c26-e4a9-4234-8ca4-f128d57ed10e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:04.209193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f28c26-e4a9-4234-8ca4-f128d57ed10e None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:04.210823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 109/209] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:04.216340 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3ddd0-2078-4ace-a2c1-f0e324270702 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:04.217199 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3ddd0-2078-4ace-a2c1-f0e324270702 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:04.217721 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3ddd0-2078-4ace-a2c1-f0e324270702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:04.223505 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb342742-9f87-491a-aef2-2ed5fad6bfd1 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:04.223799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb342742-9f87-491a-aef2-2ed5fad6bfd1 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:04.224084 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb342742-9f87-491a-aef2-2ed5fad6bfd1 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:04.225433 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 110/210] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:34:04.231112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd77a1-455d-45e0-8ada-5325dc6a8601 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:04.231651 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd77a1-455d-45e0-8ada-5325dc6a8601 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:04.232136 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd77a1-455d-45e0-8ada-5325dc6a8601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:04.236913 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-3bb3ddd0-2078-4ace-a2c1-f0e324270702 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:04.238284 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3bb3ddd0-2078-4ace-a2c1-f0e324270702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:04.251592 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-a1cd77a1-455d-45e0-8ada-5325dc6a8601 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:04.253238 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-a1cd77a1-455d-45e0-8ada-5325dc6a8601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:04.270708 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:04.315016 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 101/211] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:04.341798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3ddd0-2078-4ace-a2c1-f0e324270702 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:04.343648 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 111/212] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:34:04.368577 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd77a1-455d-45e0-8ada-5325dc6a8601 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:04.453113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3ddd0-2078-4ace-a2c1-f0e324270702 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:04.475932 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd77a1-455d-45e0-8ada-5325dc6a8601 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:04.491620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f9b80-7833-44fa-8cc1-685cbf90454b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:04.492672 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f9b80-7833-44fa-8cc1-685cbf90454b None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:04.493177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f9b80-7833-44fa-8cc1-685cbf90454b None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:04.493721 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f9b80-7833-44fa-8cc1-685cbf90454b None admin] PATH_INFO: `/v3/roles/member` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:04.495286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:04.507938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f9b80-7833-44fa-8cc1-685cbf90454b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:04.514003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1f7b8-d720-4d45-9f40-3a700e92a90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:04.515044 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1f7b8-d720-4d45-9f40-3a700e92a90e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:04.515516 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1f7b8-d720-4d45-9f40-3a700e92a90e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:04.515937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1f7b8-d720-4d45-9f40-3a700e92a90e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:04.517273 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1f7b8-d720-4d45-9f40-3a700e92a90e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:04.518515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-2c9f9b80-7833-44fa-8cc1-685cbf90454b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:04.518515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:04.518515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:04.518515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:04.518515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:04.518515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:04.518515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:04.518515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:04.518515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:04.518515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:04.518515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:04.518515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:04.518515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:04.518515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:04.518515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:04.518515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:04.519407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:04.519407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:04.519407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:04.519407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:04.519407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:04.519407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:04.519407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:04.519407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:04.519407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:04.519407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:04.519407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:04.519407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:04.519407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:04.519407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:04.519407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:04.519407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:04.520191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:04.520191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:04.520191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:04.520191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:04.520191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:04.520191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:04.520191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:04.520191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:04.520191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:04.520191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:04.520191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:04.520191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:04.520191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 102/213] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:34:04.521733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1f7b8-d720-4d45-9f40-3a700e92a90e None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:04.523636 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 112/214] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:04.530241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f9b80-7833-44fa-8cc1-685cbf90454b None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:04.530775 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1f7b8-d720-4d45-9f40-3a700e92a90e None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:04.562355 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:04.643592 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1f7b8-d720-4d45-9f40-3a700e92a90e None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:04.647277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f9b80-7833-44fa-8cc1-685cbf90454b None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:04.683710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f9b17-0ca1-4bca-a52b-1d70c69f2967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:04.684776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f9b17-0ca1-4bca-a52b-1d70c69f2967 None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:04.685281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f9b17-0ca1-4bca-a52b-1d70c69f2967 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:04.685806 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f9b17-0ca1-4bca-a52b-1d70c69f2967 None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:04.686728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f9b17-0ca1-4bca-a52b-1d70c69f2967 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:04.687498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b4205-6f9e-4bf0-a166-bdd17b62eb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:04.688186 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b4205-6f9e-4bf0-a166-bdd17b62eb37 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:04.688473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b4205-6f9e-4bf0-a166-bdd17b62eb37 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:04.688753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b4205-6f9e-4bf0-a166-bdd17b62eb37 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:04.689259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f9b17-0ca1-4bca-a52b-1d70c69f2967 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:04.689513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b4205-6f9e-4bf0-a166-bdd17b62eb37 None admin] RBAC: Adding query filter params (name=member) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:04.689850 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b4205-6f9e-4bf0-a166-bdd17b62eb37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:04.691346 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b4205-6f9e-4bf0-a166-bdd17b62eb37 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:04.701678 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 103/215] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:04.707430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b4205-6f9e-4bf0-a166-bdd17b62eb37 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:04.721815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 113/216] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jun 27 15:34:04 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:04.816950 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:04.832407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b4205-6f9e-4bf0-a166-bdd17b62eb37 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:04.871040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b383e-213f-41d1-9fc4-770ec3a87012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:04.871942 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b383e-213f-41d1-9fc4-770ec3a87012 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:04.872214 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b383e-213f-41d1-9fc4-770ec3a87012 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:04.872494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b383e-213f-41d1-9fc4-770ec3a87012 None admin] PATH_INFO: `/v3/users/101d1fa9b0f44cc6930c6bf1a6ba1d8a` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:04.888013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b383e-213f-41d1-9fc4-770ec3a87012 None admin] RBAC: Authorizing `identity:get_user(user_id=101d1fa9b0f44cc6930c6bf1a6ba1d8a)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:04.889513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b383e-213f-41d1-9fc4-770ec3a87012 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:04.893771 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 104/217] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:04 2022] GET /identity/v3/users/101d1fa9b0f44cc6930c6bf1a6ba1d8a => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:04.896891 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f9b17-0ca1-4bca-a52b-1d70c69f2967 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:04.931275 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c3422-bcf6-4c97-8e90-084157939d6f None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:04.931564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c3422-bcf6-4c97-8e90-084157939d6f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:04.931845 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c3422-bcf6-4c97-8e90-084157939d6f None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:04.933367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 105/218] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:34:04.938614 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91371d58-d56e-43d3-be87-8b79bb9d9f51 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:04.938907 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91371d58-d56e-43d3-be87-8b79bb9d9f51 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:04.939887 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91371d58-d56e-43d3-be87-8b79bb9d9f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:04.948930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:04.957340 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-91371d58-d56e-43d3-be87-8b79bb9d9f51 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:04.959264 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-91371d58-d56e-43d3-be87-8b79bb9d9f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:05.035266 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f9b17-0ca1-4bca-a52b-1d70c69f2967 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:05.060147 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 106/219] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:34:05.086813 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b7ab9-7258-4886-bd1c-ff2b80992aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:05.088112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b7ab9-7258-4886-bd1c-ff2b80992aea None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:05.088389 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b7ab9-7258-4886-bd1c-ff2b80992aea None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:05.088662 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b7ab9-7258-4886-bd1c-ff2b80992aea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:05.093975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91371d58-d56e-43d3-be87-8b79bb9d9f51 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:05.190131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b7ab9-7258-4886-bd1c-ff2b80992aea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:05.191857 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b7ab9-7258-4886-bd1c-ff2b80992aea None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:05.210621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:05.240304 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91371d58-d56e-43d3-be87-8b79bb9d9f51 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:05.257947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:05.298110 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d900b9-e49d-45c1-92c6-d43eddcd6ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:05.299547 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d900b9-e49d-45c1-92c6-d43eddcd6ca3 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:05.300309 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d900b9-e49d-45c1-92c6-d43eddcd6ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:05.301729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d900b9-e49d-45c1-92c6-d43eddcd6ca3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:05.318699 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d900b9-e49d-45c1-92c6-d43eddcd6ca3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:05.330860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-f4d900b9-e49d-45c1-92c6-d43eddcd6ca3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:34:05.330860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:05.330860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:05.330860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:05.330860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:05.330860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:05.330860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:05.330860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:05.330860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:05.330860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:05.330860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:05.330860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:05.330860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:05.330860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:05.330860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:05.330860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:05.331826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:05.331826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:05.331826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:05.331826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:05.331826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:05.331826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:05.331826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:05.331826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:05.331826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:05.331826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:05.331826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:05.331826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:05.331826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:05.331826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:05.331826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:05.331826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:05.332635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:05.332635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:05.332635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:05.332635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:05.332635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:05.332635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:05.332635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:05.332635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:05.332635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:05.332635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:05.332635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:34:05.332635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:05.335083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 107/221] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:34:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:05.335483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 114/221] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:05.341735 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9a4d7-d4bc-49e5-866d-de3870d3b9f6 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:05.341885 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b7ab9-7258-4886-bd1c-ff2b80992aea None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:05.342305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9a4d7-d4bc-49e5-866d-de3870d3b9f6 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:05.342407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9a4d7-d4bc-49e5-866d-de3870d3b9f6 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:05.345912 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 108/222] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:05 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:05.347012 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9a4d7-d4bc-49e5-866d-de3870d3b9f6 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:05.468905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b7ab9-7258-4886-bd1c-ff2b80992aea None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:05.489509 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9a4d7-d4bc-49e5-866d-de3870d3b9f6 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:05.519879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917069d6-bfb4-4fe0-82e7-f3f07d207b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:05.521177 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917069d6-bfb4-4fe0-82e7-f3f07d207b1d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:05.521658 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917069d6-bfb4-4fe0-82e7-f3f07d207b1d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:05.522419 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917069d6-bfb4-4fe0-82e7-f3f07d207b1d None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:05.524213 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917069d6-bfb4-4fe0-82e7-f3f07d207b1d None admin] RBAC: Authorizing `identity:get_project(project_id=9974edfe18e14b8a9711b3f5c02808a9)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:05.527255 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917069d6-bfb4-4fe0-82e7-f3f07d207b1d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:05.527702 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978b508-834c-4e73-a086-b33960dff197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:05.528541 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978b508-834c-4e73-a086-b33960dff197 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:05.528967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978b508-834c-4e73-a086-b33960dff197 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:05.529345 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978b508-834c-4e73-a086-b33960dff197 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:05.529917 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 115/223] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:05 2022] GET /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:05.530645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c978b508-834c-4e73-a086-b33960dff197 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:05.531055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c978b508-834c-4e73-a086-b33960dff197 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:05.531657 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad287f-af11-44a3-81d3-c9722303f2d6 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:05.532081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad287f-af11-44a3-81d3-c9722303f2d6 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:05.532494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad287f-af11-44a3-81d3-c9722303f2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:05.532881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c978b508-834c-4e73-a086-b33960dff197 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:05.536038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:05.553161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 109/224] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:05.553161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978b508-834c-4e73-a086-b33960dff197 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:05.558427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-52ad287f-af11-44a3-81d3-c9722303f2d6 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:05.559308 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-52ad287f-af11-44a3-81d3-c9722303f2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:05.562734 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:05.638084 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 116/225] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:05.639208 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad287f-af11-44a3-81d3-c9722303f2d6 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:05.696563 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978b508-834c-4e73-a086-b33960dff197 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:05.745695 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110e001-eb7c-4877-a5d1-dfd4f115408e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:05.746788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110e001-eb7c-4877-a5d1-dfd4f115408e None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:05.747780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110e001-eb7c-4877-a5d1-dfd4f115408e None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:05.748479 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110e001-eb7c-4877-a5d1-dfd4f115408e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:05.749506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110e001-eb7c-4877-a5d1-dfd4f115408e None admin] RBAC: Adding query filter params (role.id=5dc71eded4f64c08b2fb5f545b63b2f6, scope.project.id=9974edfe18e14b8a9711b3f5c02808a9, user.id=101d1fa9b0f44cc6930c6bf1a6ba1d8a) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:05.749986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110e001-eb7c-4877-a5d1-dfd4f115408e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:05.750596 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad287f-af11-44a3-81d3-c9722303f2d6 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:05.751803 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110e001-eb7c-4877-a5d1-dfd4f115408e None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:05.759969 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 110/226] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:34:05 2022] GET /identity/v3/role_assignments?user.id=101d1fa9b0f44cc6930c6bf1a6ba1d8a&scope.project.id=9974edfe18e14b8a9711b3f5c02808a9&role.id=5dc71eded4f64c08b2fb5f545b63b2f6 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:05.761124 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110e001-eb7c-4877-a5d1-dfd4f115408e None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:05.816942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e665c3-1e6a-4b49-a167-8af2cf277b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:05.818255 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e665c3-1e6a-4b49-a167-8af2cf277b6b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:05.818690 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e665c3-1e6a-4b49-a167-8af2cf277b6b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:05.819108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e665c3-1e6a-4b49-a167-8af2cf277b6b None admin] PATH_INFO: `/v3/users/7f3816d6eade43a5a50b8d7b60acc26d` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:05.820616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e665c3-1e6a-4b49-a167-8af2cf277b6b None admin] RBAC: Authorizing `identity:get_user(user_id=7f3816d6eade43a5a50b8d7b60acc26d)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:05.822813 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e665c3-1e6a-4b49-a167-8af2cf277b6b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:05.836203 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 117/227] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:05 2022] GET /identity/v3/users/7f3816d6eade43a5a50b8d7b60acc26d => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:05.843915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e665c3-1e6a-4b49-a167-8af2cf277b6b None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:05.854040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:05.872259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:05.898517 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110e001-eb7c-4877-a5d1-dfd4f115408e None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:05.940011 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae1495-e504-4902-9563-d7150d9e88e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:05.940865 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae1495-e504-4902-9563-d7150d9e88e3 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:05.941154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae1495-e504-4902-9563-d7150d9e88e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:05.941432 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae1495-e504-4902-9563-d7150d9e88e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:05.954788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae1495-e504-4902-9563-d7150d9e88e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:05.963566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-daae1495-e504-4902-9563-d7150d9e88e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:05.963566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:05.963566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:05.963566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:05.963566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:05.963566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:05.963566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:05.963566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:05.963566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:05.963566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:05.963566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:05.963566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:05.963566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:05.963566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:05.963566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:05.963566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:05.964530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:05.964530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:05.964530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:05.964530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:05.964530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:05.964530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:05.964530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:05.964530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:05.964530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:05.964530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:05.964530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:05.964530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:05.964530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:05.964530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:05.964530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:05.964530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:05.965317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:05.965317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:05.965317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:05.965317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:05.965317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:05.965317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:05.965317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:05.965317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:05.965317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:05.965317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:05.965317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:05.965317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:05.965317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 111/228] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:34:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:05.971214 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae1495-e504-4902-9563-d7150d9e88e3 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:06.078528 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e665c3-1e6a-4b49-a167-8af2cf277b6b None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:06.134589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae1495-e504-4902-9563-d7150d9e88e3 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:06.145437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57551b82-cd9a-4222-87ce-f02ce37ea2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:06.146400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57551b82-cd9a-4222-87ce-f02ce37ea2f5 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:06.146690 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57551b82-cd9a-4222-87ce-f02ce37ea2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:06.146970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57551b82-cd9a-4222-87ce-f02ce37ea2f5 None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:06.148453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57551b82-cd9a-4222-87ce-f02ce37ea2f5 None admin] RBAC: Authorizing `identity:get_project(project_id=9974edfe18e14b8a9711b3f5c02808a9)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:06.150603 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57551b82-cd9a-4222-87ce-f02ce37ea2f5 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:06.152889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 118/229] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:05 2022] GET /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9 => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:06.158318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57551b82-cd9a-4222-87ce-f02ce37ea2f5 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:06.183389 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:06.207402 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca645fd-bc39-424f-8a3c-aa86661df1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:06.208090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca645fd-bc39-424f-8a3c-aa86661df1b8 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:06.208377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca645fd-bc39-424f-8a3c-aa86661df1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:06.208656 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca645fd-bc39-424f-8a3c-aa86661df1b8 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:06.209422 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca645fd-bc39-424f-8a3c-aa86661df1b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:06.209758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca645fd-bc39-424f-8a3c-aa86661df1b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:06.211180 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca645fd-bc39-424f-8a3c-aa86661df1b8 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:06.234404 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 112/230] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:06.241484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca645fd-bc39-424f-8a3c-aa86661df1b8 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:06.322316 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:06.351865 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57551b82-cd9a-4222-87ce-f02ce37ea2f5 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:06.391794 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c66696-df44-4656-bc41-5374015deba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:06.392691 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c66696-df44-4656-bc41-5374015deba9 None admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:06.392969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c66696-df44-4656-bc41-5374015deba9 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:06.393241 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c66696-df44-4656-bc41-5374015deba9 None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9/users/7f3816d6eade43a5a50b8d7b60acc26d/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:06.395656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c66696-df44-4656-bc41-5374015deba9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9974edfe18e14b8a9711b3f5c02808a9, user_id=7f3816d6eade43a5a50b8d7b60acc26d, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:06.397085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c66696-df44-4656-bc41-5374015deba9 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:06.411571 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 119/231] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:34:06 2022] PUT /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9/users/7f3816d6eade43a5a50b8d7b60acc26d/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:06.449542 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca645fd-bc39-424f-8a3c-aa86661df1b8 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:06.507738 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158da38-9309-4560-880f-64e3f365e824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:06.508632 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158da38-9309-4560-880f-64e3f365e824 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:06.509479 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158da38-9309-4560-880f-64e3f365e824 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:06.509830 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158da38-9309-4560-880f-64e3f365e824 None admin] PATH_INFO: `/v3/users/7f3816d6eade43a5a50b8d7b60acc26d` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:06.511069 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158da38-9309-4560-880f-64e3f365e824 None admin] RBAC: Authorizing `identity:get_user(user_id=7f3816d6eade43a5a50b8d7b60acc26d)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:06.512548 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158da38-9309-4560-880f-64e3f365e824 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:06.514600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 113/232] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:06 2022] GET /identity/v3/users/7f3816d6eade43a5a50b8d7b60acc26d => generated 283 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:06.519970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c66696-df44-4656-bc41-5374015deba9 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:06.659391 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c66696-df44-4656-bc41-5374015deba9 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:06.698928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dc862-9912-4b0a-aaf3-c13f3487083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:06.700125 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dc862-9912-4b0a-aaf3-c13f3487083a None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:06.700634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dc862-9912-4b0a-aaf3-c13f3487083a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:06.701110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dc862-9912-4b0a-aaf3-c13f3487083a None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:06.702569 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201dc862-9912-4b0a-aaf3-c13f3487083a None admin] RBAC: Authorizing `identity:get_project(project_id=9974edfe18e14b8a9711b3f5c02808a9)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:06.704228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201dc862-9912-4b0a-aaf3-c13f3487083a None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:06.706518 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 120/233] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:06 2022] GET /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:06.713356 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158da38-9309-4560-880f-64e3f365e824 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:06.787643 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:06.864700 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158da38-9309-4560-880f-64e3f365e824 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:06.900509 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b8af2-0565-4f79-8b0c-9ce5b54404f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:06.901417 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b8af2-0565-4f79-8b0c-9ce5b54404f6 None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:06.901735 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b8af2-0565-4f79-8b0c-9ce5b54404f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:06.902019 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b8af2-0565-4f79-8b0c-9ce5b54404f6 None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9/users/7f3816d6eade43a5a50b8d7b60acc26d/roles/478b9cd589e846588025a4b4ed2c07f0` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:06.904330 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b8af2-0565-4f79-8b0c-9ce5b54404f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9974edfe18e14b8a9711b3f5c02808a9, user_id=7f3816d6eade43a5a50b8d7b60acc26d, role_id=478b9cd589e846588025a4b4ed2c07f0)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:06.905900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b8af2-0565-4f79-8b0c-9ce5b54404f6 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:06.923057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 114/234] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:34:06 2022] PUT /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9/users/7f3816d6eade43a5a50b8d7b60acc26d/roles/478b9cd589e846588025a4b4ed2c07f0 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:07.356663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2df3b-7159-45b9-ba59-2127bd593673 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:07.357181 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2df3b-7159-45b9-ba59-2127bd593673 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:07.357255 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2df3b-7159-45b9-ba59-2127bd593673 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:07.358820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 121/235] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:07.364114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc09fc-fb32-4703-a4f2-a6e455a4e839 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:07.364917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc09fc-fb32-4703-a4f2-a6e455a4e839 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:07.365340 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc09fc-fb32-4703-a4f2-a6e455a4e839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:07.388427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-1ffc09fc-fb32-4703-a4f2-a6e455a4e839 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:07.389883 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1ffc09fc-fb32-4703-a4f2-a6e455a4e839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:07.447102 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa471e3a-53c6-4498-811d-3342c01c47b2 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:07.447750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa471e3a-53c6-4498-811d-3342c01c47b2 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:07.448235 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa471e3a-53c6-4498-811d-3342c01c47b2 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:07.449847 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 122/236] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:07.455123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad17bb9-1c25-4557-b823-8a50368d3c00 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:07.455673 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad17bb9-1c25-4557-b823-8a50368d3c00 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:07.456174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad17bb9-1c25-4557-b823-8a50368d3c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:07.474263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-aad17bb9-1c25-4557-b823-8a50368d3c00 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:07.475846 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-aad17bb9-1c25-4557-b823-8a50368d3c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:07.482325 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 115/237] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:07.495268 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:07.522923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc09fc-fb32-4703-a4f2-a6e455a4e839 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:07.551409 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 123/238] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:34:07.578556 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad17bb9-1c25-4557-b823-8a50368d3c00 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:07.608917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:07.691515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad17bb9-1c25-4557-b823-8a50368d3c00 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:07.700175 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc09fc-fb32-4703-a4f2-a6e455a4e839 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:07.731883 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452dcf54-86da-4e8a-a9de-f885f9c5b513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:07.733050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452dcf54-86da-4e8a-a9de-f885f9c5b513 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:07.733555 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452dcf54-86da-4e8a-a9de-f885f9c5b513 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:07.734062 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452dcf54-86da-4e8a-a9de-f885f9c5b513 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:07.735474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452dcf54-86da-4e8a-a9de-f885f9c5b513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:07.737140 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452dcf54-86da-4e8a-a9de-f885f9c5b513 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:07.739295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 124/239] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:07.744927 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452dcf54-86da-4e8a-a9de-f885f9c5b513 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:07.748008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:07.767734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fc10b-8766-4a01-be42-524a9936f9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:07.768610 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fc10b-8766-4a01-be42-524a9936f9be None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:07.768899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fc10b-8766-4a01-be42-524a9936f9be None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:07.769170 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fc10b-8766-4a01-be42-524a9936f9be None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:07.792892 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fc10b-8766-4a01-be42-524a9936f9be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:07.808262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-ad7fc10b-8766-4a01-be42-524a9936f9be None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:34:07.808262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:07.808262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:07.808262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:07.808262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:07.808262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:07.808262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:07.808262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:07.808262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:07.808262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:07.808262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:07.808262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:07.808262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:07.808262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:07.808262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:07.808262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:07.809407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:07.809407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:07.809407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:07.809407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:07.809407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:07.809407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:07.809407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:07.809407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:07.809407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:07.809407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:07.809407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:07.809407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:07.809407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:07.809407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:07.809407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:07.809407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:07.810437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:07.810437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:07.810437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:07.810437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:07.810437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:07.810437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:07.810437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:07.810437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:07.810437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:07.810437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:07.810437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:34:07.810437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:07.819105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 116/240] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:34:07.825548 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fc10b-8766-4a01-be42-524a9936f9be None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:07.905335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452dcf54-86da-4e8a-a9de-f885f9c5b513 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:07.950650 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ddcbd4-a243-421a-8b09-f0ebf645f598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:07.951751 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ddcbd4-a243-421a-8b09-f0ebf645f598 None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:07.952240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ddcbd4-a243-421a-8b09-f0ebf645f598 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:07.952719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ddcbd4-a243-421a-8b09-f0ebf645f598 None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:07.953716 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ddcbd4-a243-421a-8b09-f0ebf645f598 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:07.955834 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ddcbd4-a243-421a-8b09-f0ebf645f598 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:07.995202 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 125/241] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jun 27 15:34:07 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:08.012921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fc10b-8766-4a01-be42-524a9936f9be None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:08.015020 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:08.076304 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac5a8a-0be3-4cc2-9072-e490d37366c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:08.077856 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac5a8a-0be3-4cc2-9072-e490d37366c6 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:08.078433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac5a8a-0be3-4cc2-9072-e490d37366c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:08.078917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac5a8a-0be3-4cc2-9072-e490d37366c6 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:08.079950 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac5a8a-0be3-4cc2-9072-e490d37366c6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:08.080461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac5a8a-0be3-4cc2-9072-e490d37366c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:08.082106 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac5a8a-0be3-4cc2-9072-e490d37366c6 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:08.100057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 117/242] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:08.106999 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ddcbd4-a243-421a-8b09-f0ebf645f598 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:08.184088 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:08.256112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ddcbd4-a243-421a-8b09-f0ebf645f598 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:08.301677 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42829d-a19f-4dad-9f7d-64a515e59e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:08.302954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42829d-a19f-4dad-9f7d-64a515e59e04 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:08.303441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42829d-a19f-4dad-9f7d-64a515e59e04 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:08.303921 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42829d-a19f-4dad-9f7d-64a515e59e04 None admin] PATH_INFO: `/v3/users/a519a60abbc64964aec9cdcbeb6e3b4d` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:08.325974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42829d-a19f-4dad-9f7d-64a515e59e04 None admin] RBAC: Authorizing `identity:get_user(user_id=a519a60abbc64964aec9cdcbeb6e3b4d)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:08.329362 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42829d-a19f-4dad-9f7d-64a515e59e04 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:08.333887 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 126/243] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:08 2022] GET /identity/v3/users/a519a60abbc64964aec9cdcbeb6e3b4d => generated 297 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:08.341529 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac5a8a-0be3-4cc2-9072-e490d37366c6 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:08.484450 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac5a8a-0be3-4cc2-9072-e490d37366c6 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:08.539538 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96690a1-be69-4f6f-ad74-06e1a93ee40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:08.540801 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96690a1-be69-4f6f-ad74-06e1a93ee40d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:08.541233 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96690a1-be69-4f6f-ad74-06e1a93ee40d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:08.541579 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96690a1-be69-4f6f-ad74-06e1a93ee40d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:08.590406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eb3d9-60fb-406b-a900-41aa08d8ebfe None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:08.591139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eb3d9-60fb-406b-a900-41aa08d8ebfe None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:08.591139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eb3d9-60fb-406b-a900-41aa08d8ebfe None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:08.592954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 127/244] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:08.593068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:08.609795 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09031448-ce85-44d1-bd72-3e3933b27dc2 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:08.610845 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09031448-ce85-44d1-bd72-3e3933b27dc2 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:08.611717 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09031448-ce85-44d1-bd72-3e3933b27dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:08.634224 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-09031448-ce85-44d1-bd72-3e3933b27dc2 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:08.635595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-09031448-ce85-44d1-bd72-3e3933b27dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:08.647402 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96690a1-be69-4f6f-ad74-06e1a93ee40d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:08.650115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96690a1-be69-4f6f-ad74-06e1a93ee40d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:08.722699 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 128/245] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:08.748681 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09031448-ce85-44d1-bd72-3e3933b27dc2 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:08.795114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 118/246] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:08.803854 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96690a1-be69-4f6f-ad74-06e1a93ee40d None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:08.808623 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:08.872681 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09031448-ce85-44d1-bd72-3e3933b27dc2 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:08.915234 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186e42d-0aef-404f-a140-4c96e2d3e26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:08.916052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186e42d-0aef-404f-a140-4c96e2d3e26d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:08.916397 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186e42d-0aef-404f-a140-4c96e2d3e26d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:08.916683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186e42d-0aef-404f-a140-4c96e2d3e26d None admin] PATH_INFO: `/v3/roles/member` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:08.931877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186e42d-0aef-404f-a140-4c96e2d3e26d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:08.934435 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:08.941602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-d186e42d-0aef-404f-a140-4c96e2d3e26d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:08.941602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:08.941602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:08.941602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:08.941602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:08.941602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:08.941602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:08.941602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:08.941602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:08.941602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:08.941602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:08.941602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:08.941602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:08.941602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:08.941602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:08.941602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:08.942867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:08.942867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:08.942867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:08.942867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:08.942867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:08.942867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:08.942867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:08.942867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:08.942867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:08.942867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:08.942867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:08.942867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:08.942867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:08.942867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:08.942867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:08.942867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:08.943823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:08.943823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:08.943823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:08.943823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:08.943823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:08.943823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:08.943823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:08.943823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:08.943823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:08.943823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:08.943823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:08.943823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:08.947056 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 129/247] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:08.953318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186e42d-0aef-404f-a140-4c96e2d3e26d None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:09.029128 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96690a1-be69-4f6f-ad74-06e1a93ee40d None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:09.069136 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186e42d-0aef-404f-a140-4c96e2d3e26d None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:09.086015 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:09.109764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cdd498-bf6e-465f-a7e6-33f16c7b3902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:09.110811 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdd498-bf6e-465f-a7e6-33f16c7b3902 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:09.111328 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdd498-bf6e-465f-a7e6-33f16c7b3902 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:09.111878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdd498-bf6e-465f-a7e6-33f16c7b3902 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:09.113002 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cdd498-bf6e-465f-a7e6-33f16c7b3902 None admin] RBAC: Adding query filter params (name=member) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:09.113529 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cdd498-bf6e-465f-a7e6-33f16c7b3902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:09.115357 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cdd498-bf6e-465f-a7e6-33f16c7b3902 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:09.125791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 130/248] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:09.131536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cdd498-bf6e-465f-a7e6-33f16c7b3902 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:09.134986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd3716-4436-4826-b0c4-7bc11cae3be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:09.139097 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd3716-4436-4826-b0c4-7bc11cae3be5 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:09.144866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd3716-4436-4826-b0c4-7bc11cae3be5 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:09.145456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd3716-4436-4826-b0c4-7bc11cae3be5 None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:09.147091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd3716-4436-4826-b0c4-7bc11cae3be5 None admin] RBAC: Authorizing `identity:get_project(project_id=9974edfe18e14b8a9711b3f5c02808a9)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:09.148810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd3716-4436-4826-b0c4-7bc11cae3be5 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:09.151613 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 119/249] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:08 2022] GET /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9 => generated 302 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:09.157281 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fe3bc-77c7-4b84-b4bf-342774545616 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:09.158861 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fe3bc-77c7-4b84-b4bf-342774545616 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:09.159364 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fe3bc-77c7-4b84-b4bf-342774545616 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:09.163316 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 120/250] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:09.165909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fe3bc-77c7-4b84-b4bf-342774545616 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:09.294669 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cdd498-bf6e-465f-a7e6-33f16c7b3902 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:09.367007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ab436-df49-46a4-b41a-756973857cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:09.369598 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ab436-df49-46a4-b41a-756973857cb6 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:09.370502 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ab436-df49-46a4-b41a-756973857cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:09.371356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ab436-df49-46a4-b41a-756973857cb6 None admin] PATH_INFO: `/v3/users/101d1fa9b0f44cc6930c6bf1a6ba1d8a` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:09.373929 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ab436-df49-46a4-b41a-756973857cb6 None admin] RBAC: Authorizing `identity:get_user(user_id=101d1fa9b0f44cc6930c6bf1a6ba1d8a)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:09.376867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ab436-df49-46a4-b41a-756973857cb6 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:09.380697 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 131/251] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:09 2022] GET /identity/v3/users/101d1fa9b0f44cc6930c6bf1a6ba1d8a => generated 297 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:09.387764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a879734-0930-42e8-8302-14310a6afdd6 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:09.388685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a879734-0930-42e8-8302-14310a6afdd6 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:09.389384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a879734-0930-42e8-8302-14310a6afdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:09.416107 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fe3bc-77c7-4b84-b4bf-342774545616 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:09.425500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-8a879734-0930-42e8-8302-14310a6afdd6 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:09.427968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8a879734-0930-42e8-8302-14310a6afdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:09.498082 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:09.501208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:09.510100 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9b13e-5e32-49b9-935b-1cda374a44ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:09.510962 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9b13e-5e32-49b9-935b-1cda374a44ab None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:09.511385 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9b13e-5e32-49b9-935b-1cda374a44ab None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:09.511815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9b13e-5e32-49b9-935b-1cda374a44ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:09.512837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9b13e-5e32-49b9-935b-1cda374a44ab None admin] RBAC: Adding query filter params (role.id=7a691f6f838a4a25a074d1537bb677a6, scope.project.id=9974edfe18e14b8a9711b3f5c02808a9, user.id=a519a60abbc64964aec9cdcbeb6e3b4d) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:09.513319 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9b13e-5e32-49b9-935b-1cda374a44ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:09.519559 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9b13e-5e32-49b9-935b-1cda374a44ab None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:09.527522 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 121/252] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:34:09 2022] GET /identity/v3/role_assignments?user.id=a519a60abbc64964aec9cdcbeb6e3b4d&scope.project.id=9974edfe18e14b8a9711b3f5c02808a9&role.id=7a691f6f838a4a25a074d1537bb677a6 => generated 259 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:09.528940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9b13e-5e32-49b9-935b-1cda374a44ab None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:09.580420 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 132/253] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:09.603620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f41f9f-bb74-48a4-8006-5ef60f2ce492 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:09.605277 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f41f9f-bb74-48a4-8006-5ef60f2ce492 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:09.606239 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f41f9f-bb74-48a4-8006-5ef60f2ce492 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:09.609546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 133/254] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:34:09.619623 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20082d-ba50-4599-97cb-dc8b728c5846 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:09.620546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20082d-ba50-4599-97cb-dc8b728c5846 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:09.621139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20082d-ba50-4599-97cb-dc8b728c5846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:09.660697 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-be20082d-ba50-4599-97cb-dc8b728c5846 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:09.660697 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-be20082d-ba50-4599-97cb-dc8b728c5846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:09.796258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9b13e-5e32-49b9-935b-1cda374a44ab None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:09.809462 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 134/255] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:09.810737 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20082d-ba50-4599-97cb-dc8b728c5846 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:09.844211 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0547f7-20aa-40b2-9772-06ddccb74ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:09.845028 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0547f7-20aa-40b2-9772-06ddccb74ae7 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:09.845317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0547f7-20aa-40b2-9772-06ddccb74ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:09.845747 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0547f7-20aa-40b2-9772-06ddccb74ae7 None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:09.847142 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0547f7-20aa-40b2-9772-06ddccb74ae7 None admin] RBAC: Authorizing `identity:get_project(project_id=9974edfe18e14b8a9711b3f5c02808a9)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:09.848720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0547f7-20aa-40b2-9772-06ddccb74ae7 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:09.851049 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 122/256] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:09 2022] GET /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9 => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:09.857767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0547f7-20aa-40b2-9772-06ddccb74ae7 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:09.954025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:09.999862 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:10.014267 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20082d-ba50-4599-97cb-dc8b728c5846 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:10.035466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0547f7-20aa-40b2-9772-06ddccb74ae7 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:10.076137 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad21d0-61d5-4e5b-b722-f898d0664443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:10.077336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad21d0-61d5-4e5b-b722-f898d0664443 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:10.077812 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad21d0-61d5-4e5b-b722-f898d0664443 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:10.078294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad21d0-61d5-4e5b-b722-f898d0664443 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:10.113285 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad21d0-61d5-4e5b-b722-f898d0664443 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:10.124490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-05ad21d0-61d5-4e5b-b722-f898d0664443 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:34:10.124490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:10.124490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:10.124490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:10.124490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:10.124490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:10.124490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:10.124490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:10.124490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:10.124490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:10.124490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:10.124490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:10.124490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:10.124490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:10.124490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:10.124490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:10.125412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:10.125412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:10.125412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:10.125412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:10.125412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:10.125412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:10.125412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:10.125412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:10.125412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:10.125412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:10.125412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:10.125412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:10.125412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:10.125412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:10.125412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:10.125412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:10.126209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:10.126209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:10.126209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:10.126209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:10.126209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:10.126209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:10.126209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:10.126209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:10.126209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:10.126209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:10.126209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:34:10.126209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:10.132405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 135/257] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:34:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:10.133881 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad21d0-61d5-4e5b-b722-f898d0664443 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:10.151124 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d16b0a-673e-45b0-8bc4-d6b847079727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:10.152482 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d16b0a-673e-45b0-8bc4-d6b847079727 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:10.152929 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d16b0a-673e-45b0-8bc4-d6b847079727 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:10.153381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d16b0a-673e-45b0-8bc4-d6b847079727 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:10.188251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d16b0a-673e-45b0-8bc4-d6b847079727 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:10.198684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-67d16b0a-673e-45b0-8bc4-d6b847079727 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:10.198684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:10.198684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:10.198684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:10.198684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:10.198684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:10.198684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:10.198684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:10.198684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:10.198684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:10.198684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:10.198684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:10.198684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:10.198684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:10.198684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:10.198684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:10.199717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:10.199717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:10.199717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:10.199717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:10.199717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:10.199717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:10.199717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:10.199717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:10.199717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:10.199717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:10.199717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:10.199717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:10.199717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:10.199717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:10.199717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:10.199717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:10.200484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:10.200484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:10.200484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:10.200484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:10.200484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:10.200484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:10.200484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:10.200484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:10.200484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:10.200484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:10.200484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:10.200484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:10.211192 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 123/258] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:34:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:34:10.217717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d16b0a-673e-45b0-8bc4-d6b847079727 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:10.327768 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad21d0-61d5-4e5b-b722-f898d0664443 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:10.409250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6e399-798f-436c-9e06-d8abc1809fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:10.410935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6e399-798f-436c-9e06-d8abc1809fb7 None admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:10.411441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6e399-798f-436c-9e06-d8abc1809fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:10.411934 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6e399-798f-436c-9e06-d8abc1809fb7 None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9/users/101d1fa9b0f44cc6930c6bf1a6ba1d8a/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:10.416184 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de6e399-798f-436c-9e06-d8abc1809fb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=9974edfe18e14b8a9711b3f5c02808a9, user_id=101d1fa9b0f44cc6930c6bf1a6ba1d8a, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:10.419222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de6e399-798f-436c-9e06-d8abc1809fb7 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:10.424335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d16b0a-673e-45b0-8bc4-d6b847079727 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:10.448622 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 136/259] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:34:10 2022] PUT /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9/users/101d1fa9b0f44cc6930c6bf1a6ba1d8a/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:10.458234 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6e399-798f-436c-9e06-d8abc1809fb7 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:10.469005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:10.492675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5ee14-d919-4d63-8a8b-0e074aabb77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:10.493827 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5ee14-d919-4d63-8a8b-0e074aabb77a None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:10.494343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5ee14-d919-4d63-8a8b-0e074aabb77a None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:10.494773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5ee14-d919-4d63-8a8b-0e074aabb77a None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:10.495711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5ee14-d919-4d63-8a8b-0e074aabb77a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:10.496164 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5ee14-d919-4d63-8a8b-0e074aabb77a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:10.496711 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:10.499555 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5ee14-d919-4d63-8a8b-0e074aabb77a None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:10.519083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 124/260] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:10.521168 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5ee14-d919-4d63-8a8b-0e074aabb77a None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:10.609826 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6e399-798f-436c-9e06-d8abc1809fb7 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:10.648930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133c8f6-22fd-4ebe-b7de-8814e9879908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:10.650369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133c8f6-22fd-4ebe-b7de-8814e9879908 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:10.650588 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133c8f6-22fd-4ebe-b7de-8814e9879908 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:10.650871 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133c8f6-22fd-4ebe-b7de-8814e9879908 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:10.651673 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133c8f6-22fd-4ebe-b7de-8814e9879908 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:10.652091 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133c8f6-22fd-4ebe-b7de-8814e9879908 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:10.653384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133c8f6-22fd-4ebe-b7de-8814e9879908 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:10.663291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 137/261] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:10.668867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133c8f6-22fd-4ebe-b7de-8814e9879908 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:10.670761 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5ee14-d919-4d63-8a8b-0e074aabb77a None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:10.714860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a060d-72da-472f-8a80-1209724c696a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:10.716002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a060d-72da-472f-8a80-1209724c696a None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:10.716425 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a060d-72da-472f-8a80-1209724c696a None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:10.716839 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a060d-72da-472f-8a80-1209724c696a None admin] PATH_INFO: `/v3/users/7f3816d6eade43a5a50b8d7b60acc26d` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:10.718385 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a060d-72da-472f-8a80-1209724c696a None admin] RBAC: Authorizing `identity:get_user(user_id=7f3816d6eade43a5a50b8d7b60acc26d)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:10.720006 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a060d-72da-472f-8a80-1209724c696a None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:10.725757 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 125/262] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:10 2022] GET /identity/v3/users/7f3816d6eade43a5a50b8d7b60acc26d => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:10.729704 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a060d-72da-472f-8a80-1209724c696a None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:10.769045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:10.793747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133c8f6-22fd-4ebe-b7de-8814e9879908 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:10.809297 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:10.832490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afe641-256d-4b15-afee-2149908e1175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:10.833707 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afe641-256d-4b15-afee-2149908e1175 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:10.834203 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afe641-256d-4b15-afee-2149908e1175 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:10.834658 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afe641-256d-4b15-afee-2149908e1175 None admin] PATH_INFO: `/v3/users/7f3816d6eade43a5a50b8d7b60acc26d` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:10.836783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5afe641-256d-4b15-afee-2149908e1175 None admin] RBAC: Authorizing `identity:get_user(user_id=7f3816d6eade43a5a50b8d7b60acc26d)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:10.840036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5afe641-256d-4b15-afee-2149908e1175 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:10.842810 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 138/263] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:10 2022] GET /identity/v3/users/7f3816d6eade43a5a50b8d7b60acc26d => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:10.848314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afe641-256d-4b15-afee-2149908e1175 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:10.874856 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a060d-72da-472f-8a80-1209724c696a None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:10.935921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05408d5-c102-48e3-b4c5-1d4d70163231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:10.937268 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05408d5-c102-48e3-b4c5-1d4d70163231 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:10.937791 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05408d5-c102-48e3-b4c5-1d4d70163231 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:10.938335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05408d5-c102-48e3-b4c5-1d4d70163231 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:10.959220 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afe641-256d-4b15-afee-2149908e1175 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:10.995471 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d28db0-1e53-4d7b-9d62-e26f6c01cb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:10.996133 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d28db0-1e53-4d7b-9d62-e26f6c01cb12 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:10.996421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d28db0-1e53-4d7b-9d62-e26f6c01cb12 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:10.996698 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d28db0-1e53-4d7b-9d62-e26f6c01cb12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:11.014512 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:11.031851 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05408d5-c102-48e3-b4c5-1d4d70163231 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:11.033553 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05408d5-c102-48e3-b4c5-1d4d70163231 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:11.077429 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d28db0-1e53-4d7b-9d62-e26f6c01cb12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:11.078948 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d28db0-1e53-4d7b-9d62-e26f6c01cb12 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:11.095734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:11.149758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 126/264] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 15:34:11.155606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05408d5-c102-48e3-b4c5-1d4d70163231 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:11.187235 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 139/265] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:11.192515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d28db0-1e53-4d7b-9d62-e26f6c01cb12 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:11.251106 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:11.302666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05408d5-c102-48e3-b4c5-1d4d70163231 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:11.317663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d28db0-1e53-4d7b-9d62-e26f6c01cb12 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:11.363770 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5d9e1-9f31-43ce-8b6b-62754a483b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:11.364921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5d9e1-9f31-43ce-8b6b-62754a483b6f None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:11.365382 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5d9e1-9f31-43ce-8b6b-62754a483b6f None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:11.365832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5d9e1-9f31-43ce-8b6b-62754a483b6f None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:11.371003 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a5d9e1-9f31-43ce-8b6b-62754a483b6f None admin] RBAC: Authorizing `identity:get_project(project_id=9974edfe18e14b8a9711b3f5c02808a9)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:11.374071 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a5d9e1-9f31-43ce-8b6b-62754a483b6f None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:11.383168 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546c28c-cbff-4f84-9ca0-422d2bde4a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:11.383168 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546c28c-cbff-4f84-9ca0-422d2bde4a6e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:11.383168 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546c28c-cbff-4f84-9ca0-422d2bde4a6e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:11.383168 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546c28c-cbff-4f84-9ca0-422d2bde4a6e None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:11.389046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546c28c-cbff-4f84-9ca0-422d2bde4a6e None admin] RBAC: Authorizing `identity:get_project(project_id=9974edfe18e14b8a9711b3f5c02808a9)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:11.389046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546c28c-cbff-4f84-9ca0-422d2bde4a6e None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:11.389046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 140/267] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:11 2022] GET /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:11.389403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 127/267] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:11 2022] GET /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:11.389698 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546c28c-cbff-4f84-9ca0-422d2bde4a6e None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:11.396238 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5d9e1-9f31-43ce-8b6b-62754a483b6f None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:11.437478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:11.575974 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5d9e1-9f31-43ce-8b6b-62754a483b6f None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:11.579638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546c28c-cbff-4f84-9ca0-422d2bde4a6e None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:11.600242 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:11.639072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ba8b7-9f29-43b8-a707-15e63134f6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:11.639731 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ba8b7-9f29-43b8-a707-15e63134f6c4 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:11.640007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ba8b7-9f29-43b8-a707-15e63134f6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:11.640284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ba8b7-9f29-43b8-a707-15e63134f6c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:11.641129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ba8b7-9f29-43b8-a707-15e63134f6c4 None admin] RBAC: Adding query filter params (role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.project.id=9974edfe18e14b8a9711b3f5c02808a9, user.id=7f3816d6eade43a5a50b8d7b60acc26d) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:11.641439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ba8b7-9f29-43b8-a707-15e63134f6c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:11.642928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ba8b7-9f29-43b8-a707-15e63134f6c4 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:11.658156 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 141/268] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:34:11 2022] GET /identity/v3/role_assignments?user.id=7f3816d6eade43a5a50b8d7b60acc26d&scope.project.id=9974edfe18e14b8a9711b3f5c02808a9&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:11.695690 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a44f45-759f-487e-a961-42988d48cec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:11.696301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a44f45-759f-487e-a961-42988d48cec5 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:11.696963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a44f45-759f-487e-a961-42988d48cec5 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:11.697254 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a44f45-759f-487e-a961-42988d48cec5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:11.698192 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a44f45-759f-487e-a961-42988d48cec5 None admin] RBAC: Adding query filter params (role.id=478b9cd589e846588025a4b4ed2c07f0, scope.project.id=9974edfe18e14b8a9711b3f5c02808a9, user.id=7f3816d6eade43a5a50b8d7b60acc26d) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:11.698514 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a44f45-759f-487e-a961-42988d48cec5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:11.700027 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a44f45-759f-487e-a961-42988d48cec5 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:11.711294 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 128/269] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:34:11 2022] GET /identity/v3/role_assignments?user.id=7f3816d6eade43a5a50b8d7b60acc26d&scope.project.id=9974edfe18e14b8a9711b3f5c02808a9&role.id=478b9cd589e846588025a4b4ed2c07f0 => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:12.077080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a182d6-bfa7-4782-a493-07c1f17955da None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:12.077280 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a182d6-bfa7-4782-a493-07c1f17955da None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:12.077530 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a182d6-bfa7-4782-a493-07c1f17955da None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:12.079420 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 142/270] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:12.085951 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94e923-8952-4a90-99dd-38d14ac30344 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:12.086214 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94e923-8952-4a90-99dd-38d14ac30344 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:12.086507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94e923-8952-4a90-99dd-38d14ac30344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:12.105468 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-8e94e923-8952-4a90-99dd-38d14ac30344 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:12.106699 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-8e94e923-8952-4a90-99dd-38d14ac30344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:12.140305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a00d7d-93b9-4272-b169-3f9206b7e2f5 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:12.140613 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a00d7d-93b9-4272-b169-3f9206b7e2f5 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:12.140858 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a00d7d-93b9-4272-b169-3f9206b7e2f5 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:12.142434 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 143/271] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:34:12.148077 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c9010-f53b-43b4-8104-1ab8fbc2c7be None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:12.148370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c9010-f53b-43b4-8104-1ab8fbc2c7be None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:12.148631 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c9010-f53b-43b4-8104-1ab8fbc2c7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:12.185476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-823c9010-f53b-43b4-8104-1ab8fbc2c7be None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:12.187576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-823c9010-f53b-43b4-8104-1ab8fbc2c7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:12.218310 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 129/272] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:34:12.245098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94e923-8952-4a90-99dd-38d14ac30344 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:12.300596 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 144/273] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:12.307754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:12.332926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c9010-f53b-43b4-8104-1ab8fbc2c7be None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:12.408237 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94e923-8952-4a90-99dd-38d14ac30344 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:12.459506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6216bde-9306-46d3-9e41-c5ded15c8511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:12.460597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6216bde-9306-46d3-9e41-c5ded15c8511 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:12.460884 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6216bde-9306-46d3-9e41-c5ded15c8511 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:12.461164 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6216bde-9306-46d3-9e41-c5ded15c8511 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:12.476943 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6216bde-9306-46d3-9e41-c5ded15c8511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:12.487438 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c9010-f53b-43b4-8104-1ab8fbc2c7be None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:12.491236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-d6216bde-9306-46d3-9e41-c5ded15c8511 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:12.491236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:12.491236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:12.491236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:12.491236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:12.491236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:12.491236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:12.491236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:12.491236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:12.491236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:12.491236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:12.491236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:12.491236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:12.491236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:12.491236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:12.491236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:12.492709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:12.492709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:12.492709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:12.492709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:12.492709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:12.492709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:12.492709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:12.492709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:12.492709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:12.492709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:12.492709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:12.492709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:12.492709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:12.492709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:12.492709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:12.492709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:12.494798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:12.494798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:12.494798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:12.494798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:12.494798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:12.494798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:12.494798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:12.494798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:12.494798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:12.494798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:12.494798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:12.494798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:12.499441 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 130/274] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:12.502059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:12.507150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6216bde-9306-46d3-9e41-c5ded15c8511 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:12.543492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56196626-d75d-4f8c-8fb3-7074753c108e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:12.544667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56196626-d75d-4f8c-8fb3-7074753c108e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:12.545161 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56196626-d75d-4f8c-8fb3-7074753c108e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:12.545703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56196626-d75d-4f8c-8fb3-7074753c108e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:12.547175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56196626-d75d-4f8c-8fb3-7074753c108e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:12.548859 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56196626-d75d-4f8c-8fb3-7074753c108e None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:12.550810 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 145/275] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:12.555419 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56104d8e-91be-4c20-8aae-b2def88f57f2 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:12.555610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56104d8e-91be-4c20-8aae-b2def88f57f2 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:12.556107 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56104d8e-91be-4c20-8aae-b2def88f57f2 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:12.558418 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 146/276] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:12.558997 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56104d8e-91be-4c20-8aae-b2def88f57f2 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:12.627277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:12.657448 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6216bde-9306-46d3-9e41-c5ded15c8511 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:12.703567 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56104d8e-91be-4c20-8aae-b2def88f57f2 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:12.710329 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b86ecd-52c9-47f1-a697-e2b8e67b258e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:12.711012 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b86ecd-52c9-47f1-a697-e2b8e67b258e None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:12.711296 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b86ecd-52c9-47f1-a697-e2b8e67b258e None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:12.711584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b86ecd-52c9-47f1-a697-e2b8e67b258e None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:12.712891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b86ecd-52c9-47f1-a697-e2b8e67b258e None admin] RBAC: Adding query filter params (name=member) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:12.713201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b86ecd-52c9-47f1-a697-e2b8e67b258e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:12.714799 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b86ecd-52c9-47f1-a697-e2b8e67b258e None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:12.729282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 131/277] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:12.729282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656b79a-2d77-4219-a419-5da85f063bed None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:12.729282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656b79a-2d77-4219-a419-5da85f063bed None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:12.729282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656b79a-2d77-4219-a419-5da85f063bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:12.744003 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-0656b79a-2d77-4219-a419-5da85f063bed None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:12.745230 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-0656b79a-2d77-4219-a419-5da85f063bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:12.750931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1cb90-4fb5-47b7-9abd-60942aa4b12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:12.751610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1cb90-4fb5-47b7-9abd-60942aa4b12f None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:12.751894 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1cb90-4fb5-47b7-9abd-60942aa4b12f None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:12.752164 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1cb90-4fb5-47b7-9abd-60942aa4b12f None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:12.752883 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1cb90-4fb5-47b7-9abd-60942aa4b12f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:12.754377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1cb90-4fb5-47b7-9abd-60942aa4b12f None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:12.779233 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 147/278] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jun 27 15:34:12 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:12.780079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1cb90-4fb5-47b7-9abd-60942aa4b12f None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:12.826038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:12.862800 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 132/279] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:12.900299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656b79a-2d77-4219-a419-5da85f063bed None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:12.945767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1cb90-4fb5-47b7-9abd-60942aa4b12f None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:12.993850 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b64ff7-ec75-4ce9-925e-46ae8720d6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:12.994738 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b64ff7-ec75-4ce9-925e-46ae8720d6f3 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:12.995029 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b64ff7-ec75-4ce9-925e-46ae8720d6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:12.995315 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b64ff7-ec75-4ce9-925e-46ae8720d6f3 None admin] PATH_INFO: `/v3/users/f70bae1e18c94173b0ae2bb8c3d71668` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:13.011968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:13.018092 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b64ff7-ec75-4ce9-925e-46ae8720d6f3 None admin] RBAC: Authorizing `identity:get_user(user_id=f70bae1e18c94173b0ae2bb8c3d71668)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:13.018835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b64ff7-ec75-4ce9-925e-46ae8720d6f3 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:13.021384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 148/280] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:12 2022] GET /identity/v3/users/f70bae1e18c94173b0ae2bb8c3d71668 => generated 293 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:13.029750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b64ff7-ec75-4ce9-925e-46ae8720d6f3 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:13.062695 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656b79a-2d77-4219-a419-5da85f063bed None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:13.137069 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be4784-e092-42d3-aed5-50088fc120f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:13.137923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be4784-e092-42d3-aed5-50088fc120f5 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:13.138233 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be4784-e092-42d3-aed5-50088fc120f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:13.138668 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be4784-e092-42d3-aed5-50088fc120f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:13.155549 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be4784-e092-42d3-aed5-50088fc120f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:13.165154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:13.166616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-73be4784-e092-42d3-aed5-50088fc120f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:34:13.166616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:13.166616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:13.166616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:13.166616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:13.166616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:13.166616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:13.166616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:13.166616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:13.166616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:13.166616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:13.166616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:13.166616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:13.166616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:13.166616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:13.166616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:13.167494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:13.167494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:13.167494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:13.167494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:13.167494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:13.167494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:13.167494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:13.167494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:13.167494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:13.167494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:13.167494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:13.167494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:13.167494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:13.167494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:13.167494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:13.167494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:13.168282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:13.168282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:13.168282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:13.168282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:13.168282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:13.168282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:13.168282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:13.168282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:13.168282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:13.168282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:13.168282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:34:13.168282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:13.169045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 133/281] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:13.184801 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be4784-e092-42d3-aed5-50088fc120f5 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:13.189266 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b64ff7-ec75-4ce9-925e-46ae8720d6f3 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:13.252893 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90aa0e7-c1ad-4e0a-9b42-aa8917b5d8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:13.254804 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90aa0e7-c1ad-4e0a-9b42-aa8917b5d8e5 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:13.255828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90aa0e7-c1ad-4e0a-9b42-aa8917b5d8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:13.256129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90aa0e7-c1ad-4e0a-9b42-aa8917b5d8e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:13.257551 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90aa0e7-c1ad-4e0a-9b42-aa8917b5d8e5 None admin] RBAC: Adding query filter params (role.id=5dc71eded4f64c08b2fb5f545b63b2f6, user.id=f70bae1e18c94173b0ae2bb8c3d71668, scope.system=all) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:13.257950 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90aa0e7-c1ad-4e0a-9b42-aa8917b5d8e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:13.261002 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90aa0e7-c1ad-4e0a-9b42-aa8917b5d8e5 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:13.269719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 149/282] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Jun 27 15:34:13 2022] GET /identity/v3/role_assignments?user.id=f70bae1e18c94173b0ae2bb8c3d71668&scope.system=all&role.id=5dc71eded4f64c08b2fb5f545b63b2f6 => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:13.337538 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be4784-e092-42d3-aed5-50088fc120f5 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:13.388425 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:13.398520 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9702919-fbbe-40b0-862b-2532c3bf3076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:13.399205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9702919-fbbe-40b0-862b-2532c3bf3076 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:13.399485 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9702919-fbbe-40b0-862b-2532c3bf3076 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:13.399948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9702919-fbbe-40b0-862b-2532c3bf3076 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:13.400723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9702919-fbbe-40b0-862b-2532c3bf3076 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:13.401026 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9702919-fbbe-40b0-862b-2532c3bf3076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:13.402471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9702919-fbbe-40b0-862b-2532c3bf3076 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:13.418278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 134/283] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:13.423452 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90aa0e7-c1ad-4e0a-9b42-aa8917b5d8e5 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:13.566145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90aa0e7-c1ad-4e0a-9b42-aa8917b5d8e5 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:13.622430 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7694255-0780-462a-ae76-d74cc499f6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:13.623701 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7694255-0780-462a-ae76-d74cc499f6da None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:13.624180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7694255-0780-462a-ae76-d74cc499f6da None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:13.624667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7694255-0780-462a-ae76-d74cc499f6da None admin] PATH_INFO: `/v3/users/a519a60abbc64964aec9cdcbeb6e3b4d` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:13.626236 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7694255-0780-462a-ae76-d74cc499f6da None admin] RBAC: Authorizing `identity:get_user(user_id=a519a60abbc64964aec9cdcbeb6e3b4d)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:13.628012 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7694255-0780-462a-ae76-d74cc499f6da None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:13.628659 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845def9-5e6e-4af4-9b7e-1f20d40eafee None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:13.629086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845def9-5e6e-4af4-9b7e-1f20d40eafee None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:13.629269 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845def9-5e6e-4af4-9b7e-1f20d40eafee None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:13.630642 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 135/284] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:13.631023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 150/285] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:13 2022] GET /identity/v3/users/a519a60abbc64964aec9cdcbeb6e3b4d => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:13.637346 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c0f30-0239-4235-8c0c-0cf89fa80fce None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:13.637627 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c0f30-0239-4235-8c0c-0cf89fa80fce None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:13.637959 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c0f30-0239-4235-8c0c-0cf89fa80fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:13.651757 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7694255-0780-462a-ae76-d74cc499f6da None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:13.656351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-553c0f30-0239-4235-8c0c-0cf89fa80fce None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:13.657842 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-553c0f30-0239-4235-8c0c-0cf89fa80fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:13.702492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:13.780001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 136/286] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:13.821661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c0f30-0239-4235-8c0c-0cf89fa80fce None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:13.839987 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7694255-0780-462a-ae76-d74cc499f6da None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:13.883217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a3343-c310-4b06-ad06-dd4e32b2b88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:13.885279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a3343-c310-4b06-ad06-dd4e32b2b88d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:13.885922 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a3343-c310-4b06-ad06-dd4e32b2b88d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:13.886562 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a3343-c310-4b06-ad06-dd4e32b2b88d None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:13.888113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a3343-c310-4b06-ad06-dd4e32b2b88d None admin] RBAC: Authorizing `identity:get_project(project_id=9974edfe18e14b8a9711b3f5c02808a9)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:13.889859 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a3343-c310-4b06-ad06-dd4e32b2b88d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:13.891850 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 151/287] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:13 2022] GET /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:13.898109 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a3343-c310-4b06-ad06-dd4e32b2b88d None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:14.050348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c0f30-0239-4235-8c0c-0cf89fa80fce None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:14.053791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:14.077755 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a3343-c310-4b06-ad06-dd4e32b2b88d None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:14.110019 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:14.145303 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c1975-fffa-42ce-b298-dc5835da19b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:14.146489 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c1975-fffa-42ce-b298-dc5835da19b6 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:14.146980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c1975-fffa-42ce-b298-dc5835da19b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:14.147468 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c1975-fffa-42ce-b298-dc5835da19b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:14.158103 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1029a-1f36-4ace-85c9-3cbfc490ee51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:14.159497 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1029a-1f36-4ace-85c9-3cbfc490ee51 None admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:14.159786 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1029a-1f36-4ace-85c9-3cbfc490ee51 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:14.160072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1029a-1f36-4ace-85c9-3cbfc490ee51 None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9/users/a519a60abbc64964aec9cdcbeb6e3b4d/roles/7a691f6f838a4a25a074d1537bb677a6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:14.163314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c1029a-1f36-4ace-85c9-3cbfc490ee51 None admin] RBAC: Authorizing `identity:create_grant(project_id=9974edfe18e14b8a9711b3f5c02808a9, user_id=a519a60abbc64964aec9cdcbeb6e3b4d, role_id=7a691f6f838a4a25a074d1537bb677a6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:14.165982 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c1029a-1f36-4ace-85c9-3cbfc490ee51 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:14.177194 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c1975-fffa-42ce-b298-dc5835da19b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:14.189518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-682c1975-fffa-42ce-b298-dc5835da19b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:14.189518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:14.189518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:14.189518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:14.189518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:14.189518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:14.189518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:14.189518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:14.189518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:14.189518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:14.189518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:14.189518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:14.189518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:14.189518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:14.189518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:14.189518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:14.190797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:14.190797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:14.190797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:14.190797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:14.190797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:14.190797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:14.190797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:14.190797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:14.190797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:14.190797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:14.190797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:14.190797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:14.190797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:14.190797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:14.190797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:14.190797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:14.191908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:14.191908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:14.191908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:14.191908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:14.191908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:14.191908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:14.191908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:14.191908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:14.191908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:14.191908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:14.191908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:14.191908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:14.202494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 152/288] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:34:13 2022] PUT /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9/users/a519a60abbc64964aec9cdcbeb6e3b4d/roles/7a691f6f838a4a25a074d1537bb677a6 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:14.203760 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 137/289] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:34:14.211475 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1029a-1f36-4ace-85c9-3cbfc490ee51 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:14.388708 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1029a-1f36-4ace-85c9-3cbfc490ee51 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:14.434487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95beee71-645b-482e-9a69-e935ffa377a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:14.435182 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95beee71-645b-482e-9a69-e935ffa377a7 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:14.435467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95beee71-645b-482e-9a69-e935ffa377a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:14.435745 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95beee71-645b-482e-9a69-e935ffa377a7 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:14.436513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95beee71-645b-482e-9a69-e935ffa377a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:14.436826 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95beee71-645b-482e-9a69-e935ffa377a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:14.439201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95beee71-645b-482e-9a69-e935ffa377a7 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:14.449491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 153/290] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:14.458272 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c1975-fffa-42ce-b298-dc5835da19b6 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:14.680731 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c1975-fffa-42ce-b298-dc5835da19b6 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:14.732913 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd11a07-5d1d-4b29-9753-cb5ff2e21b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:14.733838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd11a07-5d1d-4b29-9753-cb5ff2e21b1b None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:14.734129 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd11a07-5d1d-4b29-9753-cb5ff2e21b1b None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:14.734416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd11a07-5d1d-4b29-9753-cb5ff2e21b1b None admin] PATH_INFO: `/v3/users/101d1fa9b0f44cc6930c6bf1a6ba1d8a` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:14.735703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd11a07-5d1d-4b29-9753-cb5ff2e21b1b None admin] RBAC: Authorizing `identity:get_user(user_id=101d1fa9b0f44cc6930c6bf1a6ba1d8a)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:14.737219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd11a07-5d1d-4b29-9753-cb5ff2e21b1b None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:14.739019 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 138/291] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:14 2022] GET /identity/v3/users/101d1fa9b0f44cc6930c6bf1a6ba1d8a => generated 297 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:14.749049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95beee71-645b-482e-9a69-e935ffa377a7 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:14.780029 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:14.907976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95beee71-645b-482e-9a69-e935ffa377a7 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:14.949498 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c262d1-cff8-4e43-ae7e-6cc3a1ddba28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:14.950092 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c262d1-cff8-4e43-ae7e-6cc3a1ddba28 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:14.950382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c262d1-cff8-4e43-ae7e-6cc3a1ddba28 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:14.950663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c262d1-cff8-4e43-ae7e-6cc3a1ddba28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:15.043010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c262d1-cff8-4e43-ae7e-6cc3a1ddba28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:15.044517 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c262d1-cff8-4e43-ae7e-6cc3a1ddba28 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:15.061121 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:15.176632 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 154/292] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:15.184641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd11a07-5d1d-4b29-9753-cb5ff2e21b1b None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:15.293156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:15.376604 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd11a07-5d1d-4b29-9753-cb5ff2e21b1b None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:15.424314 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1bc010-b237-437f-ae56-9d591cfb602e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:15.425221 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bc010-b237-437f-ae56-9d591cfb602e None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:15.425514 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bc010-b237-437f-ae56-9d591cfb602e None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:15.425831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bc010-b237-437f-ae56-9d591cfb602e None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:15.427140 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bc010-b237-437f-ae56-9d591cfb602e None admin] RBAC: Authorizing `identity:get_project(project_id=9974edfe18e14b8a9711b3f5c02808a9)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:15.428698 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bc010-b237-437f-ae56-9d591cfb602e None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:15.431339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 139/293] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:15 2022] GET /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:15.440191 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c262d1-cff8-4e43-ae7e-6cc3a1ddba28 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:15.670047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c262d1-cff8-4e43-ae7e-6cc3a1ddba28 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:15.705130 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:15.757504 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5812755c-6e0c-40af-b1f6-ed091a84f94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:15.758630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812755c-6e0c-40af-b1f6-ed091a84f94e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:15.759647 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812755c-6e0c-40af-b1f6-ed091a84f94e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:15.760950 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812755c-6e0c-40af-b1f6-ed091a84f94e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:15.762568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5812755c-6e0c-40af-b1f6-ed091a84f94e None admin] RBAC: Adding query filter params (role.id=5dc71eded4f64c08b2fb5f545b63b2f6, scope.project.id=9974edfe18e14b8a9711b3f5c02808a9, user.id=101d1fa9b0f44cc6930c6bf1a6ba1d8a) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:15.763121 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5812755c-6e0c-40af-b1f6-ed091a84f94e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:15.765828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5812755c-6e0c-40af-b1f6-ed091a84f94e None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:15.778545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 155/294] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:34:15 2022] GET /identity/v3/role_assignments?user.id=101d1fa9b0f44cc6930c6bf1a6ba1d8a&scope.project.id=9974edfe18e14b8a9711b3f5c02808a9&role.id=5dc71eded4f64c08b2fb5f545b63b2f6 => generated 608 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:16.007465 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f71cb-00fb-4b61-9249-f1079bcb59a5 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:16.007966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f71cb-00fb-4b61-9249-f1079bcb59a5 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:16.007966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f71cb-00fb-4b61-9249-f1079bcb59a5 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:16.010391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 140/295] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:16.016966 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d86ab8-3f97-4ce5-a72a-be3fa79cfeba None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:16.017604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d86ab8-3f97-4ce5-a72a-be3fa79cfeba None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:16.017786 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d86ab8-3f97-4ce5-a72a-be3fa79cfeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:16.042830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-43d86ab8-3f97-4ce5-a72a-be3fa79cfeba None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:16.044095 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-43d86ab8-3f97-4ce5-a72a-be3fa79cfeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:16.128259 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6781bcb-0970-4ae6-b0f1-9d1ae205e92e None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:16.128665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6781bcb-0970-4ae6-b0f1-9d1ae205e92e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:16.129076 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6781bcb-0970-4ae6-b0f1-9d1ae205e92e None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:16.131074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 141/296] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:16.143531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e637a-20ac-497b-b755-16dddd352e85 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:16.143945 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e637a-20ac-497b-b755-16dddd352e85 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:16.144422 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 156/297] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:16.144589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e637a-20ac-497b-b755-16dddd352e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:16.168160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d86ab8-3f97-4ce5-a72a-be3fa79cfeba None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:16.184143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-b30e637a-20ac-497b-b755-16dddd352e85 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:16.186328 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-b30e637a-20ac-497b-b755-16dddd352e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:16.304507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 142/298] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:16.306354 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a46b9-c24b-4cb1-aba9-09eac06b61cb None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:16.306838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a46b9-c24b-4cb1-aba9-09eac06b61cb None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:16.307293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a46b9-c24b-4cb1-aba9-09eac06b61cb None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:16.308890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 143/299] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:16.313538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d86ab8-3f97-4ce5-a72a-be3fa79cfeba None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:16.315117 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7f613-2a6f-4663-81be-67b511ab47e8 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:16.315613 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7f613-2a6f-4663-81be-67b511ab47e8 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:16.316109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7f613-2a6f-4663-81be-67b511ab47e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:16.320075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:16.337968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-2cd7f613-2a6f-4663-81be-67b511ab47e8 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:16.339826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-2cd7f613-2a6f-4663-81be-67b511ab47e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:16.357392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fc764-521f-4180-b838-b421368962b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:16.358235 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fc764-521f-4180-b838-b421368962b4 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:16.358528 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fc764-521f-4180-b838-b421368962b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:16.358806 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fc764-521f-4180-b838-b421368962b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:16.372288 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368fc764-521f-4180-b838-b421368962b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:16.384417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-368fc764-521f-4180-b838-b421368962b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:16.384417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:16.384417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:16.384417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:16.384417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:16.384417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:16.384417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:16.384417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:16.384417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:16.384417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:16.384417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:16.384417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:16.384417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:16.384417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:16.384417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:16.384417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:16.385741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:16.385741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:16.385741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:16.385741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:16.385741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:16.385741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:16.385741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:16.385741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:16.385741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:16.385741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:16.385741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:16.385741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:16.385741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:16.385741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:16.385741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:16.385741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:16.387101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:16.387101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:16.387101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:16.387101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:16.387101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:16.387101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:16.387101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:16.387101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:16.387101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:16.387101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:16.387101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:16.387101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:16.387101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 157/300] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:16.390009 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fc764-521f-4180-b838-b421368962b4 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:16.395595 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:16.429698 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 144/301] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:16.429698 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7f613-2a6f-4663-81be-67b511ab47e8 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:16.542461 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fc764-521f-4180-b838-b421368962b4 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:16.549919 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7f613-2a6f-4663-81be-67b511ab47e8 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:16.586707 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ad047-0a64-4d04-8178-7899598b6c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:16.587762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ad047-0a64-4d04-8178-7899598b6c36 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:16.588187 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ad047-0a64-4d04-8178-7899598b6c36 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:16.588468 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ad047-0a64-4d04-8178-7899598b6c36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:16.589766 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ad047-0a64-4d04-8178-7899598b6c36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:16.591225 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ad047-0a64-4d04-8178-7899598b6c36 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:16.593709 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 158/302] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:16.593709 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ad047-0a64-4d04-8178-7899598b6c36 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:16.606574 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bcf04-4595-4f6b-a409-2fc689c916e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:16.608102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bcf04-4595-4f6b-a409-2fc689c916e6 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:16.608102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bcf04-4595-4f6b-a409-2fc689c916e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:16.608394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bcf04-4595-4f6b-a409-2fc689c916e6 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:16.609491 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1bcf04-4595-4f6b-a409-2fc689c916e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:16.609935 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1bcf04-4595-4f6b-a409-2fc689c916e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:16.612146 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1bcf04-4595-4f6b-a409-2fc689c916e6 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:16.626149 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 145/303] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:16.626940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bcf04-4595-4f6b-a409-2fc689c916e6 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:16.643598 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:16.704473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:16.751195 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ad047-0a64-4d04-8178-7899598b6c36 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:16.784232 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bcf04-4595-4f6b-a409-2fc689c916e6 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:16.801516 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9488543-1ad5-4b3e-bb4f-633fd1aaca3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:16.803322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9488543-1ad5-4b3e-bb4f-633fd1aaca3b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:16.803852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9488543-1ad5-4b3e-bb4f-633fd1aaca3b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:16.804143 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9488543-1ad5-4b3e-bb4f-633fd1aaca3b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:16.828739 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9488543-1ad5-4b3e-bb4f-633fd1aaca3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:16.841971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-f9488543-1ad5-4b3e-bb4f-633fd1aaca3b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:34:16.841971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:16.841971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:16.841971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:16.841971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:16.841971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:16.841971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:16.841971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:16.841971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:16.841971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:16.841971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:16.841971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:16.841971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:16.841971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:16.841971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:16.841971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:16.843250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:16.843250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:16.843250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:16.843250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:16.843250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:16.843250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:16.843250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:16.843250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:16.843250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:16.843250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:16.843250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:16.843250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:16.843250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:16.843250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:16.843250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:16.843250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:16.844356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:16.844356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:16.844356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:16.844356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:16.844356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:16.844356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:16.844356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:16.844356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:16.844356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:16.844356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:16.844356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:34:16.844356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:16.844356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 159/304] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:16.846254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9488543-1ad5-4b3e-bb4f-633fd1aaca3b None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:16.857220 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c39abf-b3d2-44ca-b027-cc4b998db10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:16.858324 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c39abf-b3d2-44ca-b027-cc4b998db10d None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:16.858793 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c39abf-b3d2-44ca-b027-cc4b998db10d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:16.859239 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c39abf-b3d2-44ca-b027-cc4b998db10d None admin] PATH_INFO: `/v3/groups` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:16.860449 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c39abf-b3d2-44ca-b027-cc4b998db10d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:16.862714 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c39abf-b3d2-44ca-b027-cc4b998db10d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:16.897788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 146/305] 10.222.0.20 () {60 vars in 1134 bytes} [Mon Jun 27 15:34:16 2022] POST /identity/v3/groups => generated 229 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:16.897788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c39abf-b3d2-44ca-b027-cc4b998db10d None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:17.031357 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:17.042453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9488543-1ad5-4b3e-bb4f-633fd1aaca3b None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:17.045775 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:17.051301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c39abf-b3d2-44ca-b027-cc4b998db10d None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:17.095783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad6631-f7d6-45a8-9f55-b91aea169b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:17.095975 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad6631-f7d6-45a8-9f55-b91aea169b0c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:17.096222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad6631-f7d6-45a8-9f55-b91aea169b0c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:17.096697 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad6631-f7d6-45a8-9f55-b91aea169b0c None admin] PATH_INFO: `/v3/users/f70bae1e18c94173b0ae2bb8c3d71668` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:17.098928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad6631-f7d6-45a8-9f55-b91aea169b0c None admin] RBAC: Authorizing `identity:get_user(user_id=f70bae1e18c94173b0ae2bb8c3d71668)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:17.102642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad6631-f7d6-45a8-9f55-b91aea169b0c None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:17.105778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 160/306] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:16 2022] GET /identity/v3/users/f70bae1e18c94173b0ae2bb8c3d71668 => generated 293 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:17.110843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad6631-f7d6-45a8-9f55-b91aea169b0c None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:17.112436 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1778e8-cf0e-4db1-a215-1fe9734580ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:17.113149 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1778e8-cf0e-4db1-a215-1fe9734580ae None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:17.113432 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1778e8-cf0e-4db1-a215-1fe9734580ae None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:17.114826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1778e8-cf0e-4db1-a215-1fe9734580ae None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:17.114826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1778e8-cf0e-4db1-a215-1fe9734580ae None admin] RBAC: Adding query filter params (name=reader) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:17.114826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1778e8-cf0e-4db1-a215-1fe9734580ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:17.116340 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1778e8-cf0e-4db1-a215-1fe9734580ae None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:17.130393 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 147/307] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:17.136880 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1778e8-cf0e-4db1-a215-1fe9734580ae None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:17.260290 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad6631-f7d6-45a8-9f55-b91aea169b0c None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:17.301270 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24a1bd-5306-4d3c-8c6d-f116c60a8ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:17.302175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24a1bd-5306-4d3c-8c6d-f116c60a8ce5 None admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:17.302460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24a1bd-5306-4d3c-8c6d-f116c60a8ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:17.302750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24a1bd-5306-4d3c-8c6d-f116c60a8ce5 None admin] PATH_INFO: `/v3/system/users/f70bae1e18c94173b0ae2bb8c3d71668/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:17.304245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24a1bd-5306-4d3c-8c6d-f116c60a8ce5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f70bae1e18c94173b0ae2bb8c3d71668, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:17.305674 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f24a1bd-5306-4d3c-8c6d-f116c60a8ce5 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:17.307094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1778e8-cf0e-4db1-a215-1fe9734580ae None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:17.315912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 161/308] 10.222.0.20 () {58 vars in 1334 bytes} [Mon Jun 27 15:34:17 2022] PUT /identity/v3/system/users/f70bae1e18c94173b0ae2bb8c3d71668/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:17.325993 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98097951-d390-4056-a757-761752d849f0 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:17.325993 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98097951-d390-4056-a757-761752d849f0 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:17.325993 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98097951-d390-4056-a757-761752d849f0 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:17.325993 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 162/309] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:17.330394 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b67a2b-5567-407c-babf-47024af7d5f3 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:17.330917 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b67a2b-5567-407c-babf-47024af7d5f3 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:17.331299 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b67a2b-5567-407c-babf-47024af7d5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:17.366947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-78b67a2b-5567-407c-babf-47024af7d5f3 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:17.370466 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-78b67a2b-5567-407c-babf-47024af7d5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:17.373525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acef7cc-e9ec-4383-b21a-243f9e34b163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:17.374519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acef7cc-e9ec-4383-b21a-243f9e34b163 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:17.374801 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acef7cc-e9ec-4383-b21a-243f9e34b163 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:17.375073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acef7cc-e9ec-4383-b21a-243f9e34b163 None admin] PATH_INFO: `/v3/users/6a9d1766cf804e8e984c43d0f7ac9e54` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:17.395953 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acef7cc-e9ec-4383-b21a-243f9e34b163 None admin] RBAC: Authorizing `identity:get_user(user_id=6a9d1766cf804e8e984c43d0f7ac9e54)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:17.398709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acef7cc-e9ec-4383-b21a-243f9e34b163 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:17.402180 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 148/310] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:17 2022] GET /identity/v3/users/6a9d1766cf804e8e984c43d0f7ac9e54 => generated 293 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:17.409162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acef7cc-e9ec-4383-b21a-243f9e34b163 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:17.422048 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:17.448051 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:17.506069 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 163/311] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:17.524416 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b67a2b-5567-407c-babf-47024af7d5f3 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:17.561913 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acef7cc-e9ec-4383-b21a-243f9e34b163 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:17.603043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016284f7-51c3-4677-a44f-6e64c0ae1b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:17.604066 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016284f7-51c3-4677-a44f-6e64c0ae1b12 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:17.604502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016284f7-51c3-4677-a44f-6e64c0ae1b12 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:17.604929 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016284f7-51c3-4677-a44f-6e64c0ae1b12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:17.606035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016284f7-51c3-4677-a44f-6e64c0ae1b12 None admin] RBAC: Adding query filter params (role.id=7a691f6f838a4a25a074d1537bb677a6, user.id=6a9d1766cf804e8e984c43d0f7ac9e54, scope.system=all) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:17.606503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016284f7-51c3-4677-a44f-6e64c0ae1b12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:17.608142 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016284f7-51c3-4677-a44f-6e64c0ae1b12 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:17.614285 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 149/312] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Jun 27 15:34:17 2022] GET /identity/v3/role_assignments?user.id=6a9d1766cf804e8e984c43d0f7ac9e54&scope.system=all&role.id=7a691f6f838a4a25a074d1537bb677a6 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:17.669766 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b67a2b-5567-407c-babf-47024af7d5f3 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:17.714520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd25a6-eabc-48a9-a29f-75ee09892ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:17.715662 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd25a6-eabc-48a9-a29f-75ee09892ecb None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:17.716118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd25a6-eabc-48a9-a29f-75ee09892ecb None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:17.716539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd25a6-eabc-48a9-a29f-75ee09892ecb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:17.734054 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd25a6-eabc-48a9-a29f-75ee09892ecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:17.745977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-39bd25a6-eabc-48a9-a29f-75ee09892ecb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:34:17.745977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:17.745977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:17.745977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:17.745977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:17.745977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:17.745977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:17.745977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:17.745977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:17.745977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:17.745977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:17.745977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:17.745977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:17.745977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:17.745977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:17.745977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:17.747110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:17.747110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:17.747110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:17.747110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:17.747110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:17.747110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:17.747110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:17.747110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:17.747110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:17.747110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:17.747110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:17.747110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:17.747110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:17.747110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:17.747110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:17.747110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:17.748078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:17.748078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:17.748078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:17.748078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:17.748078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:17.748078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:17.748078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:17.748078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:17.748078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:17.748078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:17.748078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:34:17.748078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:17.751226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 164/313] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:17.757459 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016284f7-51c3-4677-a44f-6e64c0ae1b12 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:17.864195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:17.887484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016284f7-51c3-4677-a44f-6e64c0ae1b12 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:17.925969 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef9b89-18f6-49bd-aea4-3fc3c0c4cdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:17.926620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef9b89-18f6-49bd-aea4-3fc3c0c4cdc4 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:17.926904 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef9b89-18f6-49bd-aea4-3fc3c0c4cdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:17.927179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef9b89-18f6-49bd-aea4-3fc3c0c4cdc4 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:17.927938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef9b89-18f6-49bd-aea4-3fc3c0c4cdc4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:17.928244 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef9b89-18f6-49bd-aea4-3fc3c0c4cdc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:17.929662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef9b89-18f6-49bd-aea4-3fc3c0c4cdc4 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:17.940094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 150/314] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:17.946960 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd25a6-eabc-48a9-a29f-75ee09892ecb None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:17.970057 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:18.062360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd25a6-eabc-48a9-a29f-75ee09892ecb None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:18.100063 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c8a3a-f390-48b8-806f-a6c1d3c83b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:18.100964 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c8a3a-f390-48b8-806f-a6c1d3c83b96 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:18.101248 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c8a3a-f390-48b8-806f-a6c1d3c83b96 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:18.101527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c8a3a-f390-48b8-806f-a6c1d3c83b96 None admin] PATH_INFO: `/v3/users/a519a60abbc64964aec9cdcbeb6e3b4d` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:18.103036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c8a3a-f390-48b8-806f-a6c1d3c83b96 None admin] RBAC: Authorizing `identity:get_user(user_id=a519a60abbc64964aec9cdcbeb6e3b4d)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:18.104616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c8a3a-f390-48b8-806f-a6c1d3c83b96 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:18.106687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 165/315] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:17 2022] GET /identity/v3/users/a519a60abbc64964aec9cdcbeb6e3b4d => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:18.111943 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef9b89-18f6-49bd-aea4-3fc3c0c4cdc4 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:18.227369 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef9b89-18f6-49bd-aea4-3fc3c0c4cdc4 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:18.269989 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d095c6f-aec7-4138-b75d-e565229f7a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:18.271062 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d095c6f-aec7-4138-b75d-e565229f7a08 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:18.271555 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d095c6f-aec7-4138-b75d-e565229f7a08 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:18.272040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d095c6f-aec7-4138-b75d-e565229f7a08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:18.291434 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:18.349765 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d095c6f-aec7-4138-b75d-e565229f7a08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:18.351629 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d095c6f-aec7-4138-b75d-e565229f7a08 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:18.465770 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 151/316] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:18.478310 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c8a3a-f390-48b8-806f-a6c1d3c83b96 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:18.567121 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:18.584756 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c8a3a-f390-48b8-806f-a6c1d3c83b96 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:18.621402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e96f52-b2f1-410b-b07b-21adf1d29715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:18.622304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e96f52-b2f1-410b-b07b-21adf1d29715 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:18.622591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e96f52-b2f1-410b-b07b-21adf1d29715 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:18.622865 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e96f52-b2f1-410b-b07b-21adf1d29715 None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:18.624117 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e96f52-b2f1-410b-b07b-21adf1d29715 None admin] RBAC: Authorizing `identity:get_project(project_id=9974edfe18e14b8a9711b3f5c02808a9)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:18.625572 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e96f52-b2f1-410b-b07b-21adf1d29715 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:18.627805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 166/317] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:18 2022] GET /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:18.633508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d095c6f-aec7-4138-b75d-e565229f7a08 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:18.686815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:18.747620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d095c6f-aec7-4138-b75d-e565229f7a08 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:18.789292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829b84e-711e-47e5-87b6-0a677a7f70b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:18.791168 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829b84e-711e-47e5-87b6-0a677a7f70b0 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:18.791935 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829b84e-711e-47e5-87b6-0a677a7f70b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:18.792449 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829b84e-711e-47e5-87b6-0a677a7f70b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:18.793556 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829b84e-711e-47e5-87b6-0a677a7f70b0 None admin] RBAC: Adding query filter params (role.id=7a691f6f838a4a25a074d1537bb677a6, scope.project.id=9974edfe18e14b8a9711b3f5c02808a9, user.id=a519a60abbc64964aec9cdcbeb6e3b4d) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:18.794131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829b84e-711e-47e5-87b6-0a677a7f70b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:18.795868 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829b84e-711e-47e5-87b6-0a677a7f70b0 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:18.803627 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 152/318] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:34:18 2022] GET /identity/v3/role_assignments?user.id=a519a60abbc64964aec9cdcbeb6e3b4d&scope.project.id=9974edfe18e14b8a9711b3f5c02808a9&role.id=7a691f6f838a4a25a074d1537bb677a6 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:19.292724 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60eca6c-325d-46bf-866d-f54eb9ec8cda None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:19.293241 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60eca6c-325d-46bf-866d-f54eb9ec8cda None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:19.293343 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60eca6c-325d-46bf-866d-f54eb9ec8cda None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:19.294991 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 167/319] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:19.300215 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f479d-da1b-44f9-a8a5-c82e6649eb1c None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:19.300630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f479d-da1b-44f9-a8a5-c82e6649eb1c None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:19.300795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f479d-da1b-44f9-a8a5-c82e6649eb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:19.318169 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-b95f479d-da1b-44f9-a8a5-c82e6649eb1c None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:19.319388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-b95f479d-da1b-44f9-a8a5-c82e6649eb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:19.393717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 153/320] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:19.418013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60eca6c-325d-46bf-866d-f54eb9ec8cda None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:19.525551 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60eca6c-325d-46bf-866d-f54eb9ec8cda None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:19.563389 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205def1-e974-4436-8155-beae8607308c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:19.564102 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205def1-e974-4436-8155-beae8607308c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:19.564386 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205def1-e974-4436-8155-beae8607308c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:19.564666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205def1-e974-4436-8155-beae8607308c None admin] PATH_INFO: `/v3/domains/default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:19.565884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205def1-e974-4436-8155-beae8607308c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:19.567333 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205def1-e974-4436-8155-beae8607308c None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:19.569117 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 168/321] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:19.574640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f479d-da1b-44f9-a8a5-c82e6649eb1c None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:19.615789 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:19.680568 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f479d-da1b-44f9-a8a5-c82e6649eb1c None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:19.717155 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfd83a-219f-4d60-b62d-1d4704381dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:19.717865 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfd83a-219f-4d60-b62d-1d4704381dbf None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:19.718155 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfd83a-219f-4d60-b62d-1d4704381dbf None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:19.718437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfd83a-219f-4d60-b62d-1d4704381dbf None admin] PATH_INFO: `/v3/groups` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:19.719142 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dfd83a-219f-4d60-b62d-1d4704381dbf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:19.720602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dfd83a-219f-4d60-b62d-1d4704381dbf None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:19.736241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 154/322] 10.222.0.20 () {60 vars in 1134 bytes} [Mon Jun 27 15:34:19 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:34:19.872685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c4bf1-478f-4f80-a740-5ee5fa200fa5 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:19.873004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c4bf1-478f-4f80-a740-5ee5fa200fa5 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:19.873247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c4bf1-478f-4f80-a740-5ee5fa200fa5 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:19.874876 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 169/323] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:19.880583 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28867b-41fa-4d06-acb0-d76dbcb15080 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:19.880902 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28867b-41fa-4d06-acb0-d76dbcb15080 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:19.881164 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28867b-41fa-4d06-acb0-d76dbcb15080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:19.899799 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-2f28867b-41fa-4d06-acb0-d76dbcb15080 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:19.900760 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-2f28867b-41fa-4d06-acb0-d76dbcb15080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:19.977277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 155/324] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:20.001502 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c4bf1-478f-4f80-a740-5ee5fa200fa5 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:20.028182 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:20.111942 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2728da-7aad-4b0d-a804-f3731705da15 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:20.112273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2728da-7aad-4b0d-a804-f3731705da15 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:20.112978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2728da-7aad-4b0d-a804-f3731705da15 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:20.114972 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 156/325] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:20.121041 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03794ae2-663c-4b9d-9aef-c31be18daa91 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:20.121334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03794ae2-663c-4b9d-9aef-c31be18daa91 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:20.121602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03794ae2-663c-4b9d-9aef-c31be18daa91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:20.139856 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c4bf1-478f-4f80-a740-5ee5fa200fa5 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:20.142652 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-03794ae2-663c-4b9d-9aef-c31be18daa91 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:20.143863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-03794ae2-663c-4b9d-9aef-c31be18daa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:20.180947 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:20.197621 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73df05f-c6ad-421d-9eb5-b61dafb0012f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:20.198489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73df05f-c6ad-421d-9eb5-b61dafb0012f None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:20.198767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73df05f-c6ad-421d-9eb5-b61dafb0012f None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:20.199044 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73df05f-c6ad-421d-9eb5-b61dafb0012f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:20.218412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73df05f-c6ad-421d-9eb5-b61dafb0012f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:20.227211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-d73df05f-c6ad-421d-9eb5-b61dafb0012f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:34:20.227211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:20.227211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:20.227211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:20.227211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:20.227211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:20.227211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:20.227211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:20.227211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:20.227211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:20.227211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:20.227211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:20.227211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:20.227211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:20.227211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:20.227211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:20.228189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:20.228189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:20.228189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:20.228189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:20.228189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:20.228189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:20.228189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:20.228189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:20.228189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:20.228189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:20.228189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:20.228189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:20.228189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:20.228189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:20.228189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:20.228189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:20.228963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:20.228963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:20.228963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:20.228963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:20.228963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:20.228963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:20.228963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:20.228963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:20.228963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:20.228963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:20.228963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:34:20.228963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:20.228963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 170/326] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:20.238283 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73df05f-c6ad-421d-9eb5-b61dafb0012f None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:20.253557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 157/327] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:20.282723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03794ae2-663c-4b9d-9aef-c31be18daa91 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:20.336904 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:20.360956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73df05f-c6ad-421d-9eb5-b61dafb0012f None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:20.401596 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce24f0-49a8-4ebf-8e00-b01bbe1d56ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:20.402671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce24f0-49a8-4ebf-8e00-b01bbe1d56ea None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:20.402963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce24f0-49a8-4ebf-8e00-b01bbe1d56ea None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:20.403244 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce24f0-49a8-4ebf-8e00-b01bbe1d56ea None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:20.404020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce24f0-49a8-4ebf-8e00-b01bbe1d56ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:20.404333 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce24f0-49a8-4ebf-8e00-b01bbe1d56ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:20.405800 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce24f0-49a8-4ebf-8e00-b01bbe1d56ea None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:20.420830 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03794ae2-663c-4b9d-9aef-c31be18daa91 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:20.421365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 171/328] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:20.422253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce24f0-49a8-4ebf-8e00-b01bbe1d56ea None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:20.472506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fed31-120a-47b9-bf1e-dbd4d3585e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:20.474271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fed31-120a-47b9-bf1e-dbd4d3585e3d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:20.475195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fed31-120a-47b9-bf1e-dbd4d3585e3d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:20.475683 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fed31-120a-47b9-bf1e-dbd4d3585e3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:20.492646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fed31-120a-47b9-bf1e-dbd4d3585e3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:20.503879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-c35fed31-120a-47b9-bf1e-dbd4d3585e3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:20.503879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:20.503879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:20.503879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:20.503879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:20.503879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:20.503879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:20.503879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:20.503879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:20.503879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:20.503879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:20.503879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:20.503879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:20.503879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:20.503879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:20.503879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:20.504812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:20.504812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:20.504812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:20.504812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:20.504812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:20.504812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:20.504812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:20.504812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:20.504812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:20.504812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:20.504812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:20.504812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:20.504812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:20.504812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:20.504812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:20.504812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:20.505596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:20.505596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:20.505596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:20.505596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:20.505596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:20.505596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:20.505596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:20.505596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:20.505596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:20.505596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:20.505596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:20.505596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:20.509246 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 158/329] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:34:20.518562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fed31-120a-47b9-bf1e-dbd4d3585e3d None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:20.547706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:20.563805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce24f0-49a8-4ebf-8e00-b01bbe1d56ea None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:20.611706 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c862633-ba57-48e9-a57e-e409baa16946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:20.613416 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c862633-ba57-48e9-a57e-e409baa16946 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:20.613936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c862633-ba57-48e9-a57e-e409baa16946 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:20.614579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c862633-ba57-48e9-a57e-e409baa16946 None admin] PATH_INFO: `/v3/users/6a9d1766cf804e8e984c43d0f7ac9e54` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:20.616093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c862633-ba57-48e9-a57e-e409baa16946 None admin] RBAC: Authorizing `identity:get_user(user_id=6a9d1766cf804e8e984c43d0f7ac9e54)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:20.617916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c862633-ba57-48e9-a57e-e409baa16946 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:20.620258 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 172/330] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:20 2022] GET /identity/v3/users/6a9d1766cf804e8e984c43d0f7ac9e54 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:20.627071 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c862633-ba57-48e9-a57e-e409baa16946 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:20.651299 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:20.677538 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fed31-120a-47b9-bf1e-dbd4d3585e3d None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:20.727201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c595436-15f1-47f6-a5a4-ce301da9ee44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:20.728253 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c595436-15f1-47f6-a5a4-ce301da9ee44 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:20.728664 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c595436-15f1-47f6-a5a4-ce301da9ee44 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:20.729045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c595436-15f1-47f6-a5a4-ce301da9ee44 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:20.730418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c595436-15f1-47f6-a5a4-ce301da9ee44 None admin] RBAC: Adding query filter params (name=member) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:20.730418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c595436-15f1-47f6-a5a4-ce301da9ee44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:20.731895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c595436-15f1-47f6-a5a4-ce301da9ee44 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:20.742719 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 159/331] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:20.749148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c595436-15f1-47f6-a5a4-ce301da9ee44 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:20.768824 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c862633-ba57-48e9-a57e-e409baa16946 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:20.827579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86926fd3-2998-4736-8746-b4068f41f18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:20.828831 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86926fd3-2998-4736-8746-b4068f41f18b None admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:20.829306 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86926fd3-2998-4736-8746-b4068f41f18b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:20.829807 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86926fd3-2998-4736-8746-b4068f41f18b None admin] PATH_INFO: `/v3/system/users/6a9d1766cf804e8e984c43d0f7ac9e54/roles/7a691f6f838a4a25a074d1537bb677a6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:20.831596 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86926fd3-2998-4736-8746-b4068f41f18b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6a9d1766cf804e8e984c43d0f7ac9e54, role_id=7a691f6f838a4a25a074d1537bb677a6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:20.838001 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86926fd3-2998-4736-8746-b4068f41f18b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:20.850732 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 173/332] 10.222.0.20 () {58 vars in 1334 bytes} [Mon Jun 27 15:34:20 2022] PUT /identity/v3/system/users/6a9d1766cf804e8e984c43d0f7ac9e54/roles/7a691f6f838a4a25a074d1537bb677a6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:20.868019 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:20.892027 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c595436-15f1-47f6-a5a4-ce301da9ee44 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:20.933217 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13a3a9-b977-4aad-a44c-847ae22067a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:20.934137 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13a3a9-b977-4aad-a44c-847ae22067a7 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:20.934426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13a3a9-b977-4aad-a44c-847ae22067a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:20.934711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13a3a9-b977-4aad-a44c-847ae22067a7 None admin] PATH_INFO: `/v3/users/f70bae1e18c94173b0ae2bb8c3d71668` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:20.936017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13a3a9-b977-4aad-a44c-847ae22067a7 None admin] RBAC: Authorizing `identity:get_user(user_id=f70bae1e18c94173b0ae2bb8c3d71668)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:20.937534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13a3a9-b977-4aad-a44c-847ae22067a7 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:20.939636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 160/333] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:20 2022] GET /identity/v3/users/f70bae1e18c94173b0ae2bb8c3d71668 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:20.945510 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86926fd3-2998-4736-8746-b4068f41f18b None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:21.060979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:21.095954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86926fd3-2998-4736-8746-b4068f41f18b None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:21.138777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2eeb9-ed4c-477b-abb2-e188a3e0add7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:21.139454 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2eeb9-ed4c-477b-abb2-e188a3e0add7 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:21.139743 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2eeb9-ed4c-477b-abb2-e188a3e0add7 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:21.140023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2eeb9-ed4c-477b-abb2-e188a3e0add7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:21.140870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2eeb9-ed4c-477b-abb2-e188a3e0add7 None admin] RBAC: Adding query filter params (role.id=5dc71eded4f64c08b2fb5f545b63b2f6, user.id=f70bae1e18c94173b0ae2bb8c3d71668, scope.system=all) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:21.141185 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2eeb9-ed4c-477b-abb2-e188a3e0add7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:21.142669 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2eeb9-ed4c-477b-abb2-e188a3e0add7 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:21.158809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 174/334] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Jun 27 15:34:20 2022] GET /identity/v3/role_assignments?user.id=f70bae1e18c94173b0ae2bb8c3d71668&scope.system=all&role.id=5dc71eded4f64c08b2fb5f545b63b2f6 => generated 510 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:22.803813 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d773e-057f-40cb-8dfa-6d53d7ca91d0 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:22.804347 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d773e-057f-40cb-8dfa-6d53d7ca91d0 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:22.804347 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d773e-057f-40cb-8dfa-6d53d7ca91d0 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:22.806105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 161/335] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:22.811585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1c81d-f3de-4dd6-95af-0b8d1e43c4c4 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:22.812063 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1c81d-f3de-4dd6-95af-0b8d1e43c4c4 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:22.812155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1c81d-f3de-4dd6-95af-0b8d1e43c4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:22.838642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-0bf1c81d-f3de-4dd6-95af-0b8d1e43c4c4 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:22.839923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0bf1c81d-f3de-4dd6-95af-0b8d1e43c4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:22.927752 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 175/336] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:34:22.954183 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d773e-057f-40cb-8dfa-6d53d7ca91d0 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:22.972681 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df7607-1e3d-41b4-b82b-a2affe130e85 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:22.972993 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df7607-1e3d-41b4-b82b-a2affe130e85 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:22.973246 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df7607-1e3d-41b4-b82b-a2affe130e85 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:22.974965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 176/337] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:34:22.982591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389950b-1c2a-4761-a8a2-839ac9e15c96 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:22.982868 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389950b-1c2a-4761-a8a2-839ac9e15c96 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:22.983154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389950b-1c2a-4761-a8a2-839ac9e15c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:23.004114 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-5389950b-1c2a-4761-a8a2-839ac9e15c96 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:23.005426 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-5389950b-1c2a-4761-a8a2-839ac9e15c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:23.074116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:23.084890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 177/338] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:34:23.127300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d773e-057f-40cb-8dfa-6d53d7ca91d0 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:23.160299 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389950b-1c2a-4761-a8a2-839ac9e15c96 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:23.176845 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c24db-a7d2-4686-8c6d-4ea85fbc31e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:23.177667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c24db-a7d2-4686-8c6d-4ea85fbc31e3 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:23.177958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c24db-a7d2-4686-8c6d-4ea85fbc31e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:23.178232 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c24db-a7d2-4686-8c6d-4ea85fbc31e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:23.200867 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c24db-a7d2-4686-8c6d-4ea85fbc31e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:23.213849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-0e6c24db-a7d2-4686-8c6d-4ea85fbc31e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:23.213849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:23.213849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:23.213849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:23.213849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:23.213849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:23.213849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:23.213849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:23.213849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:23.213849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:23.213849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:23.213849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:23.213849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:23.213849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:23.213849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:23.213849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:23.214819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:23.214819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:23.214819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:23.214819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:23.214819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:23.214819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:23.214819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:23.214819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:23.214819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:23.214819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:23.214819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:23.214819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:23.214819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:23.214819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:23.214819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:23.214819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:23.215620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:23.215620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:23.215620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:23.215620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:23.215620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:23.215620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:23.215620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:23.215620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:23.215620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:23.215620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:23.215620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:23.215620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:23.215620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 162/339] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:23.235079 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c24db-a7d2-4686-8c6d-4ea85fbc31e3 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:23.259720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:23.319895 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389950b-1c2a-4761-a8a2-839ac9e15c96 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:23.371844 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5db66-f063-4890-ad44-8ef65595e38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:23.372682 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5db66-f063-4890-ad44-8ef65595e38b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:23.372966 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5db66-f063-4890-ad44-8ef65595e38b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:23.373244 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5db66-f063-4890-ad44-8ef65595e38b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:23.393023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c24db-a7d2-4686-8c6d-4ea85fbc31e3 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:23.398573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5db66-f063-4890-ad44-8ef65595e38b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:23.409557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-72d5db66-f063-4890-ad44-8ef65595e38b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:23.409557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:23.409557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:23.409557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:23.409557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:23.409557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:23.409557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:23.409557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:23.409557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:23.409557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:23.409557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:23.409557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:23.409557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:23.409557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:23.409557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:23.409557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:23.414637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:23.414637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:23.414637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:23.414637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:23.414637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:23.414637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:23.414637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:23.414637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:23.414637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:23.414637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:23.414637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:23.414637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:23.414637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:23.414637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:23.414637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:23.414637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:23.415532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:23.415532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:23.415532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:23.415532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:23.415532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:23.415532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:23.415532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:23.415532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:23.415532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:23.415532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:23.415532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:23.415532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:23.417908 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 178/340] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:34:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:23.419427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f9a27-8b64-4e56-8198-25259cf26609 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:23.419850 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f9a27-8b64-4e56-8198-25259cf26609 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:23.420269 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f9a27-8b64-4e56-8198-25259cf26609 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:23.421941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 179/341] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:23.427398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae4340-ebcb-443d-bef4-8600d4184638 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:23.427870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae4340-ebcb-443d-bef4-8600d4184638 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:23.428287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae4340-ebcb-443d-bef4-8600d4184638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:23.455226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6a210-368d-4193-a083-4f74397ae042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:23.455963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6a210-368d-4193-a083-4f74397ae042 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:23.456257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6a210-368d-4193-a083-4f74397ae042 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:23.456542 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6a210-368d-4193-a083-4f74397ae042 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:23.457306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6a210-368d-4193-a083-4f74397ae042 None admin] RBAC: Adding query filter params (name=member) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:23.457621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6a210-368d-4193-a083-4f74397ae042 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:23.462326 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6a210-368d-4193-a083-4f74397ae042 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:23.466675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-32ae4340-ebcb-443d-bef4-8600d4184638 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:23.468092 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-32ae4340-ebcb-443d-bef4-8600d4184638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:23.481745 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 163/342] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:23.484574 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6a210-368d-4193-a083-4f74397ae042 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:23.550176 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:23.584493 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 180/343] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:34:23.585347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae4340-ebcb-443d-bef4-8600d4184638 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:23.605135 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:23.656338 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6a210-368d-4193-a083-4f74397ae042 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:23.721463 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79af20f-dfd4-4bd3-b053-23942ca46706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:23.722161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79af20f-dfd4-4bd3-b053-23942ca46706 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:23.722443 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79af20f-dfd4-4bd3-b053-23942ca46706 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:23.722715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79af20f-dfd4-4bd3-b053-23942ca46706 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:23.723470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79af20f-dfd4-4bd3-b053-23942ca46706 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:23.723775 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79af20f-dfd4-4bd3-b053-23942ca46706 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:23.725166 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79af20f-dfd4-4bd3-b053-23942ca46706 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:23.744447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 164/344] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:23.745326 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79af20f-dfd4-4bd3-b053-23942ca46706 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:23.794995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae4340-ebcb-443d-bef4-8600d4184638 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:23.889115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfe31a-8e23-47c3-98d3-59bfe0457da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:23.890018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfe31a-8e23-47c3-98d3-59bfe0457da6 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:23.890127 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfe31a-8e23-47c3-98d3-59bfe0457da6 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:23.890740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfe31a-8e23-47c3-98d3-59bfe0457da6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:23.984908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79af20f-dfd4-4bd3-b053-23942ca46706 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:23.994263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:24.006762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cfe31a-8e23-47c3-98d3-59bfe0457da6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.008580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cfe31a-8e23-47c3-98d3-59bfe0457da6 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.021252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:24.027286 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0f7c4-6277-423a-b8d9-62fbf987ccf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.028339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0f7c4-6277-423a-b8d9-62fbf987ccf2 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.028767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0f7c4-6277-423a-b8d9-62fbf987ccf2 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.029184 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0f7c4-6277-423a-b8d9-62fbf987ccf2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.046739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0f7c4-6277-423a-b8d9-62fbf987ccf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:24.057533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-a3e0f7c4-6277-423a-b8d9-62fbf987ccf2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:24.057533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:24.057533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:24.057533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:24.057533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:24.057533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:24.057533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:24.057533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:24.057533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:24.057533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:24.057533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:24.057533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:24.057533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:24.057533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:24.057533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:24.057533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:24.058709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:24.058709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:24.058709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:24.058709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:24.058709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:24.058709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:24.058709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:24.058709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:24.058709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:24.058709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:24.058709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:24.058709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:24.058709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:24.058709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:24.058709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:24.058709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:24.059570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:24.059570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:24.059570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:24.059570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:24.059570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:24.059570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:24.059570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:24.059570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:24.059570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:24.059570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:24.059570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:24.059570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:24.061987 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 165/345] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:24.063863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0f7c4-6277-423a-b8d9-62fbf987ccf2 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.225721 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 181/346] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:24.227509 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105fded-3f6b-4658-a1ec-8866216ee0a6 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.228150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105fded-3f6b-4658-a1ec-8866216ee0a6 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.229093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105fded-3f6b-4658-a1ec-8866216ee0a6 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.231399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 182/347] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:24.232427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105fded-3f6b-4658-a1ec-8866216ee0a6 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.256711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0f7c4-6277-423a-b8d9-62fbf987ccf2 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.354737 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7621818-4373-425f-97b8-ae3f0a11fb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.355415 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7621818-4373-425f-97b8-ae3f0a11fb8b None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.355697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7621818-4373-425f-97b8-ae3f0a11fb8b None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.355971 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7621818-4373-425f-97b8-ae3f0a11fb8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.360392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:24.389995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105fded-3f6b-4658-a1ec-8866216ee0a6 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.440022 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:24.442381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e685ec3-dfad-40af-b40c-943027d3937d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.443643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e685ec3-dfad-40af-b40c-943027d3937d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.443643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e685ec3-dfad-40af-b40c-943027d3937d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.443860 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e685ec3-dfad-40af-b40c-943027d3937d None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.444465 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e685ec3-dfad-40af-b40c-943027d3937d None admin] RBAC: Adding query filter params (name=member) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:24.444775 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e685ec3-dfad-40af-b40c-943027d3937d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.446235 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e685ec3-dfad-40af-b40c-943027d3937d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.456968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7621818-4373-425f-97b8-ae3f0a11fb8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.457287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 183/348] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:24.458291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e685ec3-dfad-40af-b40c-943027d3937d None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.459397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7621818-4373-425f-97b8-ae3f0a11fb8b None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.609805 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 166/349] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:24.612206 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949d433-ea75-4126-aed4-02fc4f978aba None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.612790 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949d433-ea75-4126-aed4-02fc4f978aba None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.613278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949d433-ea75-4126-aed4-02fc4f978aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.615601 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e685ec3-dfad-40af-b40c-943027d3937d None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.641857 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-0949d433-ea75-4126-aed4-02fc4f978aba None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:24.642754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-0949d433-ea75-4126-aed4-02fc4f978aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:24.668345 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10f555-4eeb-43f9-b802-bb7b00cdea36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.669680 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10f555-4eeb-43f9-b802-bb7b00cdea36 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.669680 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10f555-4eeb-43f9-b802-bb7b00cdea36 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.669888 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10f555-4eeb-43f9-b802-bb7b00cdea36 None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.671019 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b10f555-4eeb-43f9-b802-bb7b00cdea36 None admin] RBAC: Authorizing `identity:get_project(project_id=c0f0739222694cedabb8f12e4ce82e9b)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.672479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b10f555-4eeb-43f9-b802-bb7b00cdea36 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.674805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 184/350] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:24 2022] GET /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:24.676316 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17f93c-d97f-48d7-b756-e1da8e2469cc None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.676750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17f93c-d97f-48d7-b756-e1da8e2469cc None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.677211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17f93c-d97f-48d7-b756-e1da8e2469cc None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.681159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 185/351] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:24.682072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17f93c-d97f-48d7-b756-e1da8e2469cc None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.698947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:24.769707 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 167/352] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:24.771411 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949d433-ea75-4126-aed4-02fc4f978aba None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.809182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:24.831641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17f93c-d97f-48d7-b756-e1da8e2469cc None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.877098 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727e94e-ba5f-4e85-9f41-b50b8e40c620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.877713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727e94e-ba5f-4e85-9f41-b50b8e40c620 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.878006 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727e94e-ba5f-4e85-9f41-b50b8e40c620 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.878292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727e94e-ba5f-4e85-9f41-b50b8e40c620 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.915178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949d433-ea75-4126-aed4-02fc4f978aba None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:24.973568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727e94e-ba5f-4e85-9f41-b50b8e40c620 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.974272 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112a411-a95b-4287-a5bd-6f63fbc4c2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:24.975136 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727e94e-ba5f-4e85-9f41-b50b8e40c620 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.975246 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112a411-a95b-4287-a5bd-6f63fbc4c2a0 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:24.975422 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112a411-a95b-4287-a5bd-6f63fbc4c2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:24.975696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112a411-a95b-4287-a5bd-6f63fbc4c2a0 None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:24.977359 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112a411-a95b-4287-a5bd-6f63fbc4c2a0 None admin] RBAC: Authorizing `identity:get_project(project_id=c0f0739222694cedabb8f12e4ce82e9b)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:24.979976 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112a411-a95b-4287-a5bd-6f63fbc4c2a0 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:24.982749 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 168/353] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:24 2022] GET /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:24.984322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112a411-a95b-4287-a5bd-6f63fbc4c2a0 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:24.998541 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:25.111972 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:25.128732 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 186/354] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:25.130029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112a411-a95b-4287-a5bd-6f63fbc4c2a0 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:25.130139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d62d3-291b-4405-aabd-3c0b95d25485 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.133231 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d62d3-291b-4405-aabd-3c0b95d25485 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.133231 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d62d3-291b-4405-aabd-3c0b95d25485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.150446 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-ae3d62d3-291b-4405-aabd-3c0b95d25485 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:25.151786 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-ae3d62d3-291b-4405-aabd-3c0b95d25485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:25.173070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344f28b-4fe5-4f05-8e4f-8452f629b39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:25.174015 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344f28b-4fe5-4f05-8e4f-8452f629b39a None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.174296 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344f28b-4fe5-4f05-8e4f-8452f629b39a None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.174573 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344f28b-4fe5-4f05-8e4f-8452f629b39a None admin] PATH_INFO: `/v3/groups/91b7d5b4243043bd9dcb0b7289081f89` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.183686 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344f28b-4fe5-4f05-8e4f-8452f629b39a None admin] RBAC: Authorizing `identity:get_group(group_id=91b7d5b4243043bd9dcb0b7289081f89)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:25.184868 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344f28b-4fe5-4f05-8e4f-8452f629b39a None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:25.186666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 169/355] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jun 27 15:34:24 2022] GET /identity/v3/groups/91b7d5b4243043bd9dcb0b7289081f89 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:25.187453 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344f28b-4fe5-4f05-8e4f-8452f629b39a None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:25.237350 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 187/356] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:25.238262 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d62d3-291b-4405-aabd-3c0b95d25485 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:25.300382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:25.333548 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344f28b-4fe5-4f05-8e4f-8452f629b39a None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:25.373868 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d62d3-291b-4405-aabd-3c0b95d25485 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:25.375750 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62abb5-a351-4296-add3-8dc89b24779d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:25.376583 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62abb5-a351-4296-add3-8dc89b24779d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.376875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62abb5-a351-4296-add3-8dc89b24779d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.377152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62abb5-a351-4296-add3-8dc89b24779d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.392433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62abb5-a351-4296-add3-8dc89b24779d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:25.401291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-5d62abb5-a351-4296-add3-8dc89b24779d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:25.401291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:25.401291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:25.401291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:25.401291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:25.401291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:25.401291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:25.401291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:25.401291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:25.401291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:25.401291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:25.401291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:25.401291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:25.401291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:25.401291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:25.401291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:25.402299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:25.402299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:25.402299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:25.402299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:25.402299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:25.402299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:25.402299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:25.402299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:25.402299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:25.402299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:25.402299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:25.402299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:25.402299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:25.402299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:25.402299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:25.402299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:25.403115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:25.403115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:25.403115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:25.403115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:25.403115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:25.403115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:25.403115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:25.403115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:25.403115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:25.403115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:25.403115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:25.403115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:25.403115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 170/357] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:34:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:25.403906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d3265-b507-4018-b7c5-6d8c2bcd00dd None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.404198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d3265-b507-4018-b7c5-6d8c2bcd00dd None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.404475 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d3265-b507-4018-b7c5-6d8c2bcd00dd None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.405848 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 171/358] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:25.406539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d3265-b507-4018-b7c5-6d8c2bcd00dd None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:25.425690 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:25.435153 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493f51e-bf36-49b2-a343-18bb5f99fe50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:25.435153 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493f51e-bf36-49b2-a343-18bb5f99fe50 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.435153 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493f51e-bf36-49b2-a343-18bb5f99fe50 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.435153 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493f51e-bf36-49b2-a343-18bb5f99fe50 None admin] PATH_INFO: `/v3/groups/91b7d5b4243043bd9dcb0b7289081f89` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.436031 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493f51e-bf36-49b2-a343-18bb5f99fe50 None admin] RBAC: Authorizing `identity:get_group(group_id=91b7d5b4243043bd9dcb0b7289081f89)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:25.437669 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493f51e-bf36-49b2-a343-18bb5f99fe50 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:25.439267 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 188/359] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jun 27 15:34:25 2022] GET /identity/v3/groups/91b7d5b4243043bd9dcb0b7289081f89 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:25.440040 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493f51e-bf36-49b2-a343-18bb5f99fe50 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:25.547337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d3265-b507-4018-b7c5-6d8c2bcd00dd None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:25.579083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493f51e-bf36-49b2-a343-18bb5f99fe50 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:25.589453 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086fcb23-33eb-493c-852d-75f654d3e259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:25.590618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fcb23-33eb-493c-852d-75f654d3e259 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.591481 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fcb23-33eb-493c-852d-75f654d3e259 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.591778 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fcb23-33eb-493c-852d-75f654d3e259 None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.593935 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086fcb23-33eb-493c-852d-75f654d3e259 None admin] RBAC: Authorizing `identity:get_project(project_id=9974edfe18e14b8a9711b3f5c02808a9)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:25.594985 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086fcb23-33eb-493c-852d-75f654d3e259 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:25.595592 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:25.596797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 172/360] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:25 2022] GET /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:25.598598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086fcb23-33eb-493c-852d-75f654d3e259 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:25.640007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44da4d-8415-42bf-bb8b-6fdc183ad759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:25.641078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44da4d-8415-42bf-bb8b-6fdc183ad759 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.641760 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44da4d-8415-42bf-bb8b-6fdc183ad759 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.642235 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44da4d-8415-42bf-bb8b-6fdc183ad759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.643577 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d44da4d-8415-42bf-bb8b-6fdc183ad759 None admin] RBAC: Adding query filter params (group.id=91b7d5b4243043bd9dcb0b7289081f89, role.id=5dc71eded4f64c08b2fb5f545b63b2f6, scope.project.id=c0f0739222694cedabb8f12e4ce82e9b) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:25.643872 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d44da4d-8415-42bf-bb8b-6fdc183ad759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:25.645301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d44da4d-8415-42bf-bb8b-6fdc183ad759 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:25.654607 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 189/361] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jun 27 15:34:25 2022] GET /identity/v3/role_assignments?group.id=91b7d5b4243043bd9dcb0b7289081f89&scope.project.id=c0f0739222694cedabb8f12e4ce82e9b&role.id=5dc71eded4f64c08b2fb5f545b63b2f6 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:25.655966 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e533d01-4705-4090-bdbf-a6adff11cd1e None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.656266 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e533d01-4705-4090-bdbf-a6adff11cd1e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.656546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e533d01-4705-4090-bdbf-a6adff11cd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.680481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-9e533d01-4705-4090-bdbf-a6adff11cd1e None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:25.682101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-9e533d01-4705-4090-bdbf-a6adff11cd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:25.745596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:25.778433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086fcb23-33eb-493c-852d-75f654d3e259 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:25.797350 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 190/362] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:25.800709 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e533d01-4705-4090-bdbf-a6adff11cd1e None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:25.838447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe32d1-6175-4339-9593-e29ec9f10152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:25.839516 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe32d1-6175-4339-9593-e29ec9f10152 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:25.839800 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe32d1-6175-4339-9593-e29ec9f10152 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:25.840262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe32d1-6175-4339-9593-e29ec9f10152 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:25.859018 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe32d1-6175-4339-9593-e29ec9f10152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:25.867860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-67fe32d1-6175-4339-9593-e29ec9f10152 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:34:25.867860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:25.867860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:25.867860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:25.867860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:25.867860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:25.867860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:25.867860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:25.867860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:25.867860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:25.867860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:25.867860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:25.867860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:25.867860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:25.867860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:25.867860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:25.869585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:25.869585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:25.869585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:25.869585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:25.869585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:25.869585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:25.869585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:25.869585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:25.869585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:25.869585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:25.869585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:25.869585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:25.869585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:25.869585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:25.869585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:25.869585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:25.871506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:25.871506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:25.871506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:25.871506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:25.871506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:25.871506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:25.871506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:25.871506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:25.871506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:25.871506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:25.871506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 15:34:25.871506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:25.871506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 173/363] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:25.871506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe32d1-6175-4339-9593-e29ec9f10152 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:25.964637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e533d01-4705-4090-bdbf-a6adff11cd1e None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:25.969914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:26.018420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe32d1-6175-4339-9593-e29ec9f10152 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.019049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61729c-0b65-4fa2-abb8-9d45abe34fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.019934 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61729c-0b65-4fa2-abb8-9d45abe34fc2 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.020129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61729c-0b65-4fa2-abb8-9d45abe34fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.020400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61729c-0b65-4fa2-abb8-9d45abe34fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.021178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61729c-0b65-4fa2-abb8-9d45abe34fc2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:26.021593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61729c-0b65-4fa2-abb8-9d45abe34fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.024176 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61729c-0b65-4fa2-abb8-9d45abe34fc2 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.033312 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 191/364] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:26.035835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61729c-0b65-4fa2-abb8-9d45abe34fc2 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.063448 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28912768-57b6-4a9e-b0f0-77bc5f157d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.064130 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28912768-57b6-4a9e-b0f0-77bc5f157d15 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.064423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28912768-57b6-4a9e-b0f0-77bc5f157d15 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.064705 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28912768-57b6-4a9e-b0f0-77bc5f157d15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.065564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28912768-57b6-4a9e-b0f0-77bc5f157d15 None admin] RBAC: Adding query filter params (group.id=91b7d5b4243043bd9dcb0b7289081f89, role.id=478b9cd589e846588025a4b4ed2c07f0, scope.project.id=c0f0739222694cedabb8f12e4ce82e9b) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:26.065916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28912768-57b6-4a9e-b0f0-77bc5f157d15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.067434 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28912768-57b6-4a9e-b0f0-77bc5f157d15 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.080540 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 174/365] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jun 27 15:34:25 2022] GET /identity/v3/role_assignments?group.id=91b7d5b4243043bd9dcb0b7289081f89&scope.project.id=c0f0739222694cedabb8f12e4ce82e9b&role.id=478b9cd589e846588025a4b4ed2c07f0 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:26.081502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28912768-57b6-4a9e-b0f0-77bc5f157d15 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.094889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:26.201730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61729c-0b65-4fa2-abb8-9d45abe34fc2 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.246427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676be29-f739-4eea-94a2-531331979181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.247230 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676be29-f739-4eea-94a2-531331979181 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.247514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676be29-f739-4eea-94a2-531331979181 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.247792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676be29-f739-4eea-94a2-531331979181 None admin] PATH_INFO: `/v3/groups/91b7d5b4243043bd9dcb0b7289081f89` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.247968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28912768-57b6-4a9e-b0f0-77bc5f157d15 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.249140 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4676be29-f739-4eea-94a2-531331979181 None admin] RBAC: Authorizing `identity:get_group(group_id=91b7d5b4243043bd9dcb0b7289081f89)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.250796 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4676be29-f739-4eea-94a2-531331979181 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.252532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 192/366] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jun 27 15:34:26 2022] GET /identity/v3/groups/91b7d5b4243043bd9dcb0b7289081f89 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:26.254279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676be29-f739-4eea-94a2-531331979181 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.295842 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988eadd-fc33-49f0-8c21-4dc0c6b176f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.296659 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988eadd-fc33-49f0-8c21-4dc0c6b176f3 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.296939 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988eadd-fc33-49f0-8c21-4dc0c6b176f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.297209 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988eadd-fc33-49f0-8c21-4dc0c6b176f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.300454 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:26.323772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988eadd-fc33-49f0-8c21-4dc0c6b176f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:26.332445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-4988eadd-fc33-49f0-8c21-4dc0c6b176f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:34:26.332445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:26.332445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:26.332445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:26.332445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:26.332445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:26.332445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:26.332445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:26.332445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:26.332445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:26.332445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:26.332445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:26.332445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:26.332445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:26.332445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:26.332445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:26.333367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:26.333367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:26.333367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:26.333367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:26.333367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:26.333367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:26.333367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:26.333367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:26.333367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:26.333367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:26.333367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:26.333367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:26.333367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:26.333367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:26.333367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:26.333367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:26.334172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:26.334172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:26.334172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:26.334172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:26.334172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:26.334172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:26.334172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:26.334172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:26.334172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:26.334172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:26.334172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:34:26.334172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:26.337670 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 175/367] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:34:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:26.338749 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988eadd-fc33-49f0-8c21-4dc0c6b176f3 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.412406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676be29-f739-4eea-94a2-531331979181 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.448840 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:26.458894 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96082cae-ddb3-47f3-b829-9ea25c65b056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.459585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96082cae-ddb3-47f3-b829-9ea25c65b056 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.459874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96082cae-ddb3-47f3-b829-9ea25c65b056 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.460154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96082cae-ddb3-47f3-b829-9ea25c65b056 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.460922 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96082cae-ddb3-47f3-b829-9ea25c65b056 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:26.461233 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96082cae-ddb3-47f3-b829-9ea25c65b056 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.463935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96082cae-ddb3-47f3-b829-9ea25c65b056 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.476582 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 193/368] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:26.477521 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96082cae-ddb3-47f3-b829-9ea25c65b056 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.492941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988eadd-fc33-49f0-8c21-4dc0c6b176f3 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.541511 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c14fa-cf62-44f3-9a8c-7b12e07a62f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.542143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c14fa-cf62-44f3-9a8c-7b12e07a62f1 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.542433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c14fa-cf62-44f3-9a8c-7b12e07a62f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.542715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c14fa-cf62-44f3-9a8c-7b12e07a62f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.611301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:26.623526 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96082cae-ddb3-47f3-b829-9ea25c65b056 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.630145 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c14fa-cf62-44f3-9a8c-7b12e07a62f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.631627 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c14fa-cf62-44f3-9a8c-7b12e07a62f1 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.669474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268de8a3-4509-4da1-a2df-68ce5428037a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.671259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268de8a3-4509-4da1-a2df-68ce5428037a None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.671701 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268de8a3-4509-4da1-a2df-68ce5428037a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.672127 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268de8a3-4509-4da1-a2df-68ce5428037a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.673121 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268de8a3-4509-4da1-a2df-68ce5428037a None admin] RBAC: Adding query filter params (group.id=91b7d5b4243043bd9dcb0b7289081f89, role.id=5dc71eded4f64c08b2fb5f545b63b2f6, scope.project.id=9974edfe18e14b8a9711b3f5c02808a9) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:26.673589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268de8a3-4509-4da1-a2df-68ce5428037a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.675739 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268de8a3-4509-4da1-a2df-68ce5428037a None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.685093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 194/369] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jun 27 15:34:26 2022] GET /identity/v3/role_assignments?group.id=91b7d5b4243043bd9dcb0b7289081f89&scope.project.id=9974edfe18e14b8a9711b3f5c02808a9&role.id=5dc71eded4f64c08b2fb5f545b63b2f6 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:26.686381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268de8a3-4509-4da1-a2df-68ce5428037a None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.707944 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:26.755758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 176/370] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:26.755758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c14fa-cf62-44f3-9a8c-7b12e07a62f1 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.846417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268de8a3-4509-4da1-a2df-68ce5428037a None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.887648 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14e7ad-3d6f-48c8-a1ff-715959e7ffa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.888344 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14e7ad-3d6f-48c8-a1ff-715959e7ffa9 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.888631 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14e7ad-3d6f-48c8-a1ff-715959e7ffa9 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.888909 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14e7ad-3d6f-48c8-a1ff-715959e7ffa9 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.889863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14e7ad-3d6f-48c8-a1ff-715959e7ffa9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:26.890187 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14e7ad-3d6f-48c8-a1ff-715959e7ffa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.891600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14e7ad-3d6f-48c8-a1ff-715959e7ffa9 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.909021 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 195/371] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:26.910734 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14e7ad-3d6f-48c8-a1ff-715959e7ffa9 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:26.913248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c14fa-cf62-44f3-9a8c-7b12e07a62f1 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:26.962437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:26.987546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9a49c-3952-45bb-ad98-5d7145fe279f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:26.988450 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9a49c-3952-45bb-ad98-5d7145fe279f None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:26.988730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9a49c-3952-45bb-ad98-5d7145fe279f None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:26.989007 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9a49c-3952-45bb-ad98-5d7145fe279f None admin] PATH_INFO: `/v3/users/6a9d1766cf804e8e984c43d0f7ac9e54` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:26.994925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9a49c-3952-45bb-ad98-5d7145fe279f None admin] RBAC: Authorizing `identity:get_user(user_id=6a9d1766cf804e8e984c43d0f7ac9e54)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:26.996434 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9a49c-3952-45bb-ad98-5d7145fe279f None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:26.998191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 177/372] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:34:26 2022] GET /identity/v3/users/6a9d1766cf804e8e984c43d0f7ac9e54 => generated 293 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:26.999038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9a49c-3952-45bb-ad98-5d7145fe279f None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:27.065683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14e7ad-3d6f-48c8-a1ff-715959e7ffa9 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:27.066854 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:27.101732 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b0473-392b-4fa8-8080-9d9ec9c00dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:27.103120 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b0473-392b-4fa8-8080-9d9ec9c00dd1 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:27.103509 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b0473-392b-4fa8-8080-9d9ec9c00dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:27.104106 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b0473-392b-4fa8-8080-9d9ec9c00dd1 None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:27.104974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b0473-392b-4fa8-8080-9d9ec9c00dd1 None admin] RBAC: Authorizing `identity:get_project(project_id=9974edfe18e14b8a9711b3f5c02808a9)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:27.106484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b0473-392b-4fa8-8080-9d9ec9c00dd1 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:27.108282 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 196/373] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:26 2022] GET /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:27.113726 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b0473-392b-4fa8-8080-9d9ec9c00dd1 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:27.160642 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9a49c-3952-45bb-ad98-5d7145fe279f None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:27.211732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b06d0-0f18-4bfd-83ed-3ca9d02681ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:27.213921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b06d0-0f18-4bfd-83ed-3ca9d02681ee None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:27.214448 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b06d0-0f18-4bfd-83ed-3ca9d02681ee None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:27.215211 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b06d0-0f18-4bfd-83ed-3ca9d02681ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:27.216462 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:27.229404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b0473-392b-4fa8-8080-9d9ec9c00dd1 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:27.267955 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710585d5-ae47-4436-be2a-f6728ef6cca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:27.268625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710585d5-ae47-4436-be2a-f6728ef6cca1 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:27.268914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710585d5-ae47-4436-be2a-f6728ef6cca1 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:27.269197 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710585d5-ae47-4436-be2a-f6728ef6cca1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:27.270084 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710585d5-ae47-4436-be2a-f6728ef6cca1 None admin] RBAC: Adding query filter params (role.id=7a691f6f838a4a25a074d1537bb677a6, user.id=6a9d1766cf804e8e984c43d0f7ac9e54, scope.system=all) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:27.270402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710585d5-ae47-4436-be2a-f6728ef6cca1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:27.272072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710585d5-ae47-4436-be2a-f6728ef6cca1 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:27.296888 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 197/374] 10.222.0.20 () {56 vars in 1308 bytes} [Mon Jun 27 15:34:27 2022] GET /identity/v3/role_assignments?user.id=6a9d1766cf804e8e984c43d0f7ac9e54&scope.system=all&role.id=7a691f6f838a4a25a074d1537bb677a6 => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:27.298284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710585d5-ae47-4436-be2a-f6728ef6cca1 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:27.322453 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b06d0-0f18-4bfd-83ed-3ca9d02681ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:27.324004 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b06d0-0f18-4bfd-83ed-3ca9d02681ee None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:27.368079 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:27.551335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710585d5-ae47-4436-be2a-f6728ef6cca1 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:27.583914 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 178/375] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:27.590358 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b06d0-0f18-4bfd-83ed-3ca9d02681ee None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:27.628705 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c2393-0e22-4834-9537-650f4c7aba61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:27.629509 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c2393-0e22-4834-9537-650f4c7aba61 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:27.629815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c2393-0e22-4834-9537-650f4c7aba61 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:27.630089 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c2393-0e22-4834-9537-650f4c7aba61 None admin] PATH_INFO: `/v3/groups/91b7d5b4243043bd9dcb0b7289081f89` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:27.631476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c2393-0e22-4834-9537-650f4c7aba61 None admin] RBAC: Authorizing `identity:get_group(group_id=91b7d5b4243043bd9dcb0b7289081f89)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:27.632914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c2393-0e22-4834-9537-650f4c7aba61 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:27.635406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 198/376] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jun 27 15:34:27 2022] GET /identity/v3/groups/91b7d5b4243043bd9dcb0b7289081f89 => generated 226 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:27.643814 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c2393-0e22-4834-9537-650f4c7aba61 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:27.683728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:27.814463 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b06d0-0f18-4bfd-83ed-3ca9d02681ee None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:27.865218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c2393-0e22-4834-9537-650f4c7aba61 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:27.889009 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755b07c-d0fb-4068-aa7a-3ba2effaf157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:27.890173 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755b07c-d0fb-4068-aa7a-3ba2effaf157 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:27.890509 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755b07c-d0fb-4068-aa7a-3ba2effaf157 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:27.890808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755b07c-d0fb-4068-aa7a-3ba2effaf157 None admin] PATH_INFO: `/v3/projects/53048925682f4806b3831f88058d3439` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:27.892047 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755b07c-d0fb-4068-aa7a-3ba2effaf157 None admin] RBAC: Authorizing `identity:get_project(project_id=53048925682f4806b3831f88058d3439)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:27.893525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755b07c-d0fb-4068-aa7a-3ba2effaf157 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:27.895871 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 179/377] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:27 2022] GET /identity/v3/projects/53048925682f4806b3831f88058d3439 => generated 344 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:27.901961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755b07c-d0fb-4068-aa7a-3ba2effaf157 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:27.905773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:27.950298 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550b8d7-2c44-44f8-84f4-f7fb1ef3a8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:27.950975 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550b8d7-2c44-44f8-84f4-f7fb1ef3a8ec None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:27.951261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550b8d7-2c44-44f8-84f4-f7fb1ef3a8ec None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:27.951540 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550b8d7-2c44-44f8-84f4-f7fb1ef3a8ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:27.952382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550b8d7-2c44-44f8-84f4-f7fb1ef3a8ec None admin] RBAC: Adding query filter params (group.id=91b7d5b4243043bd9dcb0b7289081f89, role.id=478b9cd589e846588025a4b4ed2c07f0, scope.project.id=9974edfe18e14b8a9711b3f5c02808a9) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:27.952696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550b8d7-2c44-44f8-84f4-f7fb1ef3a8ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:27.954193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550b8d7-2c44-44f8-84f4-f7fb1ef3a8ec None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:27.967082 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 199/378] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jun 27 15:34:27 2022] GET /identity/v3/role_assignments?group.id=91b7d5b4243043bd9dcb0b7289081f89&scope.project.id=9974edfe18e14b8a9711b3f5c02808a9&role.id=478b9cd589e846588025a4b4ed2c07f0 => generated 260 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:28.058684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755b07c-d0fb-4068-aa7a-3ba2effaf157 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:28.122254 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114aebe-7112-433b-8d80-deb0af58fc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:28.123534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114aebe-7112-433b-8d80-deb0af58fc48 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:28.124031 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114aebe-7112-433b-8d80-deb0af58fc48 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:28.124538 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114aebe-7112-433b-8d80-deb0af58fc48 None admin] PATH_INFO: `/v3/groups/e93c0b5734e0471d8abaf3d06e72a5b9` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:28.142581 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114aebe-7112-433b-8d80-deb0af58fc48 None admin] RBAC: Authorizing `identity:get_group(group_id=e93c0b5734e0471d8abaf3d06e72a5b9)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:28.144455 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114aebe-7112-433b-8d80-deb0af58fc48 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:28.148392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 180/379] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jun 27 15:34:27 2022] GET /identity/v3/groups/e93c0b5734e0471d8abaf3d06e72a5b9 => generated 229 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:28.154053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550b8d7-2c44-44f8-84f4-f7fb1ef3a8ec None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:28.257775 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620e34a-0067-4306-b9b5-add23d42be80 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:28.258219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620e34a-0067-4306-b9b5-add23d42be80 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:28.258593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620e34a-0067-4306-b9b5-add23d42be80 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:28.261525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 181/380] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:28.267082 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f06da1-b480-4f6e-b442-01ef9283c2b4 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:28.267445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f06da1-b480-4f6e-b442-01ef9283c2b4 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:28.267822 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f06da1-b480-4f6e-b442-01ef9283c2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:28.275917 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:28.296768 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-98f06da1-b480-4f6e-b442-01ef9283c2b4 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:28.298296 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-98f06da1-b480-4f6e-b442-01ef9283c2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:28.308892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550b8d7-2c44-44f8-84f4-f7fb1ef3a8ec None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:28.370603 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a3050-84d8-4628-815b-f3baf127546d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:28.371278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a3050-84d8-4628-815b-f3baf127546d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:28.371569 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a3050-84d8-4628-815b-f3baf127546d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:28.371845 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a3050-84d8-4628-815b-f3baf127546d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:28.372690 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a3050-84d8-4628-815b-f3baf127546d None admin] RBAC: Adding query filter params (group.id=e93c0b5734e0471d8abaf3d06e72a5b9, role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.project.id=53048925682f4806b3831f88058d3439) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:28.373007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a3050-84d8-4628-815b-f3baf127546d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:28.374501 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a3050-84d8-4628-815b-f3baf127546d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:28.388284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 200/381] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jun 27 15:34:28 2022] GET /identity/v3/role_assignments?group.id=e93c0b5734e0471d8abaf3d06e72a5b9&scope.project.id=53048925682f4806b3831f88058d3439&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:28.390894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:28.409623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 182/382] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:34:28.434854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a3050-84d8-4628-815b-f3baf127546d None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:28.536885 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba496e-82e6-437a-b5f4-a48c71c78d8f None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:28.537235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba496e-82e6-437a-b5f4-a48c71c78d8f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:28.537437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba496e-82e6-437a-b5f4-a48c71c78d8f None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:28.539403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 183/383] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:28.545403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9ec10-4ce3-4276-a761-66209d9af366 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:28.545701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9ec10-4ce3-4276-a761-66209d9af366 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:28.547453 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9ec10-4ce3-4276-a761-66209d9af366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:28.575790 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-e3c9ec10-4ce3-4276-a761-66209d9af366 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:28.577215 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e3c9ec10-4ce3-4276-a761-66209d9af366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:28.644187 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a3050-84d8-4628-815b-f3baf127546d None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:28.678512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 184/384] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:34:28.705179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9ec10-4ce3-4276-a761-66209d9af366 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:28.724344 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e34d48-9b6f-444f-81b5-ae972936e50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:28.725207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e34d48-9b6f-444f-81b5-ae972936e50a None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:28.725500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e34d48-9b6f-444f-81b5-ae972936e50a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:28.725850 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e34d48-9b6f-444f-81b5-ae972936e50a None admin] PATH_INFO: `/v3/roles/member` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:28.757793 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e34d48-9b6f-444f-81b5-ae972936e50a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:28.774079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-c6e34d48-9b6f-444f-81b5-ae972936e50a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:28.774079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:28.774079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:28.774079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:28.774079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:28.774079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:28.774079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:28.774079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:28.774079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:28.774079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:28.774079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:28.774079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:28.774079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:28.774079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:28.774079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:28.774079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:28.775545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:28.775545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:28.775545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:28.775545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:28.775545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:28.775545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:28.775545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:28.775545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:28.775545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:28.775545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:28.775545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:28.775545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:28.775545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:28.775545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:28.775545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:28.775545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:28.776320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:28.776320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:28.776320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:28.776320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:28.776320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:28.776320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:28.776320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:28.776320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:28.776320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:28.776320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:28.776320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:28.776320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:28.780674 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 201/385] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:28.786445 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e34d48-9b6f-444f-81b5-ae972936e50a None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:28.796875 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:28.886545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9ec10-4ce3-4276-a761-66209d9af366 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:28.938092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:28.949680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567f3a1-a807-4920-9580-18df2a80bda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:28.950804 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567f3a1-a807-4920-9580-18df2a80bda6 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:28.951292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567f3a1-a807-4920-9580-18df2a80bda6 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:28.951752 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567f3a1-a807-4920-9580-18df2a80bda6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:28.972368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567f3a1-a807-4920-9580-18df2a80bda6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:28.988977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-7567f3a1-a807-4920-9580-18df2a80bda6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:28.988977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:28.988977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:28.988977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:28.988977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:28.988977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:28.988977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:28.988977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:28.988977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:28.988977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:28.988977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:28.988977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:28.988977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:28.988977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:28.988977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:28.988977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:28.989978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:28.989978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:28.989978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:28.989978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:28.989978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:28.989978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:28.989978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:28.989978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:28.989978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:28.989978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:28.989978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:28.989978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:28.989978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:28.989978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:28.989978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:28.989978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:28.990741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:28.990741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:28.990741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:28.990741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:28.990741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:28.990741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:28.990741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:28.990741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:28.990741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:28.990741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:28.990741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:28.990741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:28.997649 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 185/386] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:34:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:34:29.003007 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567f3a1-a807-4920-9580-18df2a80bda6 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.008163 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e34d48-9b6f-444f-81b5-ae972936e50a None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.091228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80075cb-3630-438f-8139-e8dd76e390cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.091913 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80075cb-3630-438f-8139-e8dd76e390cb None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.092196 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80075cb-3630-438f-8139-e8dd76e390cb None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.092472 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80075cb-3630-438f-8139-e8dd76e390cb None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.093232 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80075cb-3630-438f-8139-e8dd76e390cb None admin] RBAC: Adding query filter params (name=member) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:29.093539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80075cb-3630-438f-8139-e8dd76e390cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.094950 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80075cb-3630-438f-8139-e8dd76e390cb None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.113895 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 202/387] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:29.120119 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80075cb-3630-438f-8139-e8dd76e390cb None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.128269 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567f3a1-a807-4920-9580-18df2a80bda6 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.169585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c1a64-cce9-4a80-870e-8f0a403ade1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.170600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c1a64-cce9-4a80-870e-8f0a403ade1a None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.171045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c1a64-cce9-4a80-870e-8f0a403ade1a None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.171578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c1a64-cce9-4a80-870e-8f0a403ade1a None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.172583 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c1a64-cce9-4a80-870e-8f0a403ade1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:29.173056 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c1a64-cce9-4a80-870e-8f0a403ade1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.174671 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c1a64-cce9-4a80-870e-8f0a403ade1a None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.186235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 186/388] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:29.191584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c1a64-cce9-4a80-870e-8f0a403ade1a None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.240290 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:29.258096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:29.308384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80075cb-3630-438f-8139-e8dd76e390cb None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.309198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c1a64-cce9-4a80-870e-8f0a403ade1a None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.348997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea887f4d-580f-44b2-89ec-1c837896fcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.350319 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea887f4d-580f-44b2-89ec-1c837896fcd7 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.350820 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea887f4d-580f-44b2-89ec-1c837896fcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.351298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea887f4d-580f-44b2-89ec-1c837896fcd7 None admin] PATH_INFO: `/v3/groups/91b7d5b4243043bd9dcb0b7289081f89` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.352705 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea887f4d-580f-44b2-89ec-1c837896fcd7 None admin] RBAC: Authorizing `identity:get_group(group_id=91b7d5b4243043bd9dcb0b7289081f89)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.355240 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea887f4d-580f-44b2-89ec-1c837896fcd7 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.357376 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 187/389] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jun 27 15:34:29 2022] GET /identity/v3/groups/91b7d5b4243043bd9dcb0b7289081f89 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:29.363886 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea887f4d-580f-44b2-89ec-1c837896fcd7 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.390274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a4bc6-adf7-42c1-88c9-0edad76be630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.391155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a4bc6-adf7-42c1-88c9-0edad76be630 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.391448 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a4bc6-adf7-42c1-88c9-0edad76be630 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.391723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a4bc6-adf7-42c1-88c9-0edad76be630 None admin] PATH_INFO: `/v3/groups/91b7d5b4243043bd9dcb0b7289081f89` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.393987 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a4bc6-adf7-42c1-88c9-0edad76be630 None admin] RBAC: Authorizing `identity:get_group(group_id=91b7d5b4243043bd9dcb0b7289081f89)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.395560 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a4bc6-adf7-42c1-88c9-0edad76be630 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.397808 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 203/390] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jun 27 15:34:29 2022] GET /identity/v3/groups/91b7d5b4243043bd9dcb0b7289081f89 => generated 226 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:29.403839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a4bc6-adf7-42c1-88c9-0edad76be630 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.482553 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea887f4d-580f-44b2-89ec-1c837896fcd7 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.501584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:29.521861 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b5942-3105-4266-b124-c5bfa24293c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.523203 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b5942-3105-4266-b124-c5bfa24293c2 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.523816 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b5942-3105-4266-b124-c5bfa24293c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.525119 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b5942-3105-4266-b124-c5bfa24293c2 None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.527361 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b5942-3105-4266-b124-c5bfa24293c2 None admin] RBAC: Authorizing `identity:get_project(project_id=c0f0739222694cedabb8f12e4ce82e9b)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.529029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b5942-3105-4266-b124-c5bfa24293c2 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.531417 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 188/391] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:29 2022] GET /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:29.538324 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b5942-3105-4266-b124-c5bfa24293c2 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.593713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a4bc6-adf7-42c1-88c9-0edad76be630 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.630235 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3f7e4-25ea-4cf8-8f95-ebe693185a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.631215 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3f7e4-25ea-4cf8-8f95-ebe693185a19 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.631579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3f7e4-25ea-4cf8-8f95-ebe693185a19 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.631935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3f7e4-25ea-4cf8-8f95-ebe693185a19 None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.633254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3f7e4-25ea-4cf8-8f95-ebe693185a19 None admin] RBAC: Authorizing `identity:get_project(project_id=c0f0739222694cedabb8f12e4ce82e9b)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.635471 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3f7e4-25ea-4cf8-8f95-ebe693185a19 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.637618 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 204/392] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:29 2022] GET /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:29.643437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3f7e4-25ea-4cf8-8f95-ebe693185a19 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:29.647738 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:29.742526 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b5942-3105-4266-b124-c5bfa24293c2 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.779439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3f7e4-25ea-4cf8-8f95-ebe693185a19 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:29.780531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126122d-03f1-4168-a730-eb03b67e9a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.781483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126122d-03f1-4168-a730-eb03b67e9a32 None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.781910 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126122d-03f1-4168-a730-eb03b67e9a32 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.782205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126122d-03f1-4168-a730-eb03b67e9a32 None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b/groups/91b7d5b4243043bd9dcb0b7289081f89/roles/478b9cd589e846588025a4b4ed2c07f0` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.784587 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126122d-03f1-4168-a730-eb03b67e9a32 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0f0739222694cedabb8f12e4ce82e9b, group_id=91b7d5b4243043bd9dcb0b7289081f89, role_id=478b9cd589e846588025a4b4ed2c07f0)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.786087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126122d-03f1-4168-a730-eb03b67e9a32 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.802424 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 189/393] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Jun 27 15:34:29 2022] PUT /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b/groups/91b7d5b4243043bd9dcb0b7289081f89/roles/478b9cd589e846588025a4b4ed2c07f0 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:29.803892 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d26cfd-1eeb-4a13-acb6-3b7d0016118d None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.804145 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d26cfd-1eeb-4a13-acb6-3b7d0016118d None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.804457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d26cfd-1eeb-4a13-acb6-3b7d0016118d None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.805946 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 190/394] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:29.811981 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204806b9-6463-4368-a39a-f420b19c3352 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.812283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204806b9-6463-4368-a39a-f420b19c3352 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.812541 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204806b9-6463-4368-a39a-f420b19c3352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.831550 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-204806b9-6463-4368-a39a-f420b19c3352 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:29.832760 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-204806b9-6463-4368-a39a-f420b19c3352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:29.832999 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce5547-cc07-457c-9815-a1969afaafd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:29.834035 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce5547-cc07-457c-9815-a1969afaafd7 None admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:29.834412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce5547-cc07-457c-9815-a1969afaafd7 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:29.835120 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce5547-cc07-457c-9815-a1969afaafd7 None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b/groups/91b7d5b4243043bd9dcb0b7289081f89/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:29.839245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce5547-cc07-457c-9815-a1969afaafd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0f0739222694cedabb8f12e4ce82e9b, group_id=91b7d5b4243043bd9dcb0b7289081f89, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:29.840875 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce5547-cc07-457c-9815-a1969afaafd7 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:29.893938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 205/395] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Jun 27 15:34:29 2022] PUT /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b/groups/91b7d5b4243043bd9dcb0b7289081f89/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:29.899014 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:29.939871 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 191/396] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:34:29.983012 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce5547-cc07-457c-9815-a1969afaafd7 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:30.065784 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:30.117217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce5547-cc07-457c-9815-a1969afaafd7 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:30.164491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3522a-c40a-4088-b3f9-083e417a5534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:30.165483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3522a-c40a-4088-b3f9-083e417a5534 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:30.165927 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3522a-c40a-4088-b3f9-083e417a5534 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:30.166207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3522a-c40a-4088-b3f9-083e417a5534 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:30.177348 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3522a-c40a-4088-b3f9-083e417a5534 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:30.186018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-a2c3522a-c40a-4088-b3f9-083e417a5534 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:30.186018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:30.186018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:30.186018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:30.186018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:30.186018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:30.186018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:30.186018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:30.186018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:30.186018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:30.186018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:30.186018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:30.186018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:30.186018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:30.186018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:30.186018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:30.186878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:30.186878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:30.186878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:30.186878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:30.186878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:30.186878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:30.186878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:30.186878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:30.186878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:30.186878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:30.186878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:30.186878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:30.186878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:30.186878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:30.186878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:30.186878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:30.187729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:30.187729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:30.187729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:30.187729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:30.187729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:30.187729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:30.187729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:30.187729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:30.187729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:30.187729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:30.187729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:30.187729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:30.190890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 206/397] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:30.193663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204806b9-6463-4368-a39a-f420b19c3352 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:30.299343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204806b9-6463-4368-a39a-f420b19c3352 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:30.337915 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3f3dd-0ad6-48f1-92d0-37113e3a3eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:30.338664 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3f3dd-0ad6-48f1-92d0-37113e3a3eed None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:30.338947 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3f3dd-0ad6-48f1-92d0-37113e3a3eed None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:30.339222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3f3dd-0ad6-48f1-92d0-37113e3a3eed None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:30.339986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3f3dd-0ad6-48f1-92d0-37113e3a3eed None admin] RBAC: Adding query filter params (name=member) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:30.340289 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3f3dd-0ad6-48f1-92d0-37113e3a3eed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:30.341702 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3f3dd-0ad6-48f1-92d0-37113e3a3eed None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:30.355590 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 192/398] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:30.356986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3522a-c40a-4088-b3f9-083e417a5534 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:30.497974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3522a-c40a-4088-b3f9-083e417a5534 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:30.523802 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:30.536911 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c00347-3fdd-4eeb-8bff-58898a11a641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:30.538073 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c00347-3fdd-4eeb-8bff-58898a11a641 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:30.538374 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c00347-3fdd-4eeb-8bff-58898a11a641 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:30.538651 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c00347-3fdd-4eeb-8bff-58898a11a641 None admin] PATH_INFO: `/v3/groups/91b7d5b4243043bd9dcb0b7289081f89` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:30.539864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c00347-3fdd-4eeb-8bff-58898a11a641 None admin] RBAC: Authorizing `identity:get_group(group_id=91b7d5b4243043bd9dcb0b7289081f89)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:30.543589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c00347-3fdd-4eeb-8bff-58898a11a641 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:30.544000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 207/399] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jun 27 15:34:30 2022] GET /identity/v3/groups/91b7d5b4243043bd9dcb0b7289081f89 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:30.549230 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3f3dd-0ad6-48f1-92d0-37113e3a3eed None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:30.650212 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:30.734201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3f3dd-0ad6-48f1-92d0-37113e3a3eed None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:30.771721 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fca402-da4f-4232-8049-750f7adbc584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:30.772612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fca402-da4f-4232-8049-750f7adbc584 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:30.772897 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fca402-da4f-4232-8049-750f7adbc584 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:30.773170 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fca402-da4f-4232-8049-750f7adbc584 None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:30.774556 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fca402-da4f-4232-8049-750f7adbc584 None admin] RBAC: Authorizing `identity:get_project(project_id=9974edfe18e14b8a9711b3f5c02808a9)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:30.776100 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fca402-da4f-4232-8049-750f7adbc584 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:30.780813 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a37497-9005-4caa-a2c2-3f90a5eb04d9 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:30.781112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a37497-9005-4caa-a2c2-3f90a5eb04d9 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:30.781393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a37497-9005-4caa-a2c2-3f90a5eb04d9 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:30.781753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 193/400] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:30 2022] GET /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:30.783092 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 208/401] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:30.784023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fca402-da4f-4232-8049-750f7adbc584 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:30.791675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236464f-adce-45e1-b9e5-b39345d406e2 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:30.791993 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236464f-adce-45e1-b9e5-b39345d406e2 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:30.792238 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236464f-adce-45e1-b9e5-b39345d406e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:30.832439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-2236464f-adce-45e1-b9e5-b39345d406e2 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:30.833673 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2236464f-adce-45e1-b9e5-b39345d406e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:30.961343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fca402-da4f-4232-8049-750f7adbc584 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:30.965144 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 209/402] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:30.966604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbc294-2c4f-4ee1-971b-39f3d8b2ebe5 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:30.966898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbc294-2c4f-4ee1-971b-39f3d8b2ebe5 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:30.967182 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfbc294-2c4f-4ee1-971b-39f3d8b2ebe5 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:30.968625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 210/403] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:30.970581 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:30.975654 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357c832-98d6-4a46-8c35-eab7883f9618 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:30.975946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357c832-98d6-4a46-8c35-eab7883f9618 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:30.976229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357c832-98d6-4a46-8c35-eab7883f9618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:30.996431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-5357c832-98d6-4a46-8c35-eab7883f9618 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:30.997680 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-5357c832-98d6-4a46-8c35-eab7883f9618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:31.029802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bf7ee-9702-4fb4-95de-95b0523568db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.030804 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bf7ee-9702-4fb4-95de-95b0523568db None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.031251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bf7ee-9702-4fb4-95de-95b0523568db None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.031537 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bf7ee-9702-4fb4-95de-95b0523568db None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9/groups/91b7d5b4243043bd9dcb0b7289081f89/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.033701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2bf7ee-9702-4fb4-95de-95b0523568db None admin] RBAC: Authorizing `identity:create_grant(project_id=9974edfe18e14b8a9711b3f5c02808a9, group_id=91b7d5b4243043bd9dcb0b7289081f89, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.035306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2bf7ee-9702-4fb4-95de-95b0523568db None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.055418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 194/404] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Jun 27 15:34:30 2022] PUT /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9/groups/91b7d5b4243043bd9dcb0b7289081f89/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:31.056730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bf7ee-9702-4fb4-95de-95b0523568db None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.097423 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 211/405] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:34:31.132496 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357c832-98d6-4a46-8c35-eab7883f9618 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.245136 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:31.260717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bf7ee-9702-4fb4-95de-95b0523568db None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.309170 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036a379-90c5-4735-b2a5-5b5838e0f080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.310041 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036a379-90c5-4735-b2a5-5b5838e0f080 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.310384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036a379-90c5-4735-b2a5-5b5838e0f080 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.310668 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036a379-90c5-4735-b2a5-5b5838e0f080 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.329307 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357c832-98d6-4a46-8c35-eab7883f9618 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.331739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0036a379-90c5-4735-b2a5-5b5838e0f080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:31.349447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-0036a379-90c5-4735-b2a5-5b5838e0f080 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:31.349447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:31.349447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:31.349447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:31.349447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:31.349447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:31.349447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:31.349447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:31.349447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:31.349447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:31.349447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:31.349447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:31.349447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:31.349447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:31.349447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:31.349447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:31.350403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:31.350403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:31.350403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:31.350403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:31.350403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:31.350403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:31.350403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:31.350403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:31.350403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:31.350403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:31.350403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:31.350403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:31.350403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:31.350403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:31.350403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:31.350403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:31.351172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:31.351172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:31.351172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:31.351172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:31.351172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:31.351172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:31.351172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:31.351172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:31.351172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:31.351172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:31.351172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:31.351172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:31.351172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 195/406] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:34:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:31.356984 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036a379-90c5-4735-b2a5-5b5838e0f080 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.399530 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6089e-14d0-4468-b160-0b21f4fd62ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.403537 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6089e-14d0-4468-b160-0b21f4fd62ad None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.403829 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6089e-14d0-4468-b160-0b21f4fd62ad None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.404108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6089e-14d0-4468-b160-0b21f4fd62ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.414205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:31.426692 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6089e-14d0-4468-b160-0b21f4fd62ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:31.442723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-18c6089e-14d0-4468-b160-0b21f4fd62ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:34:31.442723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:31.442723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:31.442723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:31.442723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:31.442723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:31.442723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:31.442723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:31.442723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:31.442723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:31.442723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:31.442723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:31.442723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:31.442723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:31.442723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:31.442723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:31.443640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:31.443640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:31.443640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:31.443640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:31.443640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:31.443640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:31.443640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:31.443640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:31.443640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:31.443640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:31.443640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:31.443640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:31.443640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:31.443640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:31.443640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:31.443640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:31.444407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:31.444407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:31.444407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:31.444407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:31.444407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:31.444407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:31.444407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:31.444407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:31.444407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:31.444407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:31.444407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:34:31.444407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:31.447914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 212/407] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:34:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:34:31.453684 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6089e-14d0-4468-b160-0b21f4fd62ad None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.554572 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036a379-90c5-4735-b2a5-5b5838e0f080 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.604996 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6089e-14d0-4468-b160-0b21f4fd62ad None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.612875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bcd22-cfc8-4c65-b6fd-bb2348e52741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.613563 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bcd22-cfc8-4c65-b6fd-bb2348e52741 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.613871 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bcd22-cfc8-4c65-b6fd-bb2348e52741 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.614242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bcd22-cfc8-4c65-b6fd-bb2348e52741 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.615005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bcd22-cfc8-4c65-b6fd-bb2348e52741 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:31.615312 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bcd22-cfc8-4c65-b6fd-bb2348e52741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.615834 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:31.617827 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bcd22-cfc8-4c65-b6fd-bb2348e52741 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.636773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 196/408] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:31.642768 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bcd22-cfc8-4c65-b6fd-bb2348e52741 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.646747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb083609-6d3b-4fe4-ad65-32dc0874c757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.647470 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb083609-6d3b-4fe4-ad65-32dc0874c757 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.647760 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb083609-6d3b-4fe4-ad65-32dc0874c757 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.648038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb083609-6d3b-4fe4-ad65-32dc0874c757 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.648801 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb083609-6d3b-4fe4-ad65-32dc0874c757 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:31.649114 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb083609-6d3b-4fe4-ad65-32dc0874c757 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.652077 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb083609-6d3b-4fe4-ad65-32dc0874c757 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.672350 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 213/409] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:31.678318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb083609-6d3b-4fe4-ad65-32dc0874c757 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.798227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb083609-6d3b-4fe4-ad65-32dc0874c757 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.835088 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82b39d-0ddc-4a85-af73-eb473ef3ae11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.836280 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82b39d-0ddc-4a85-af73-eb473ef3ae11 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.836777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82b39d-0ddc-4a85-af73-eb473ef3ae11 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.836888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:31.837895 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82b39d-0ddc-4a85-af73-eb473ef3ae11 None admin] PATH_INFO: `/v3/groups/e93c0b5734e0471d8abaf3d06e72a5b9` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.839489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82b39d-0ddc-4a85-af73-eb473ef3ae11 None admin] RBAC: Authorizing `identity:get_group(group_id=e93c0b5734e0471d8abaf3d06e72a5b9)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.841124 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82b39d-0ddc-4a85-af73-eb473ef3ae11 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.843388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 214/410] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jun 27 15:34:31 2022] GET /identity/v3/groups/e93c0b5734e0471d8abaf3d06e72a5b9 => generated 229 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:31.849390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82b39d-0ddc-4a85-af73-eb473ef3ae11 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:31.872728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bcd22-cfc8-4c65-b6fd-bb2348e52741 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:31.929604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:31.943392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6ec98-62d9-40ac-a24b-ad1e75d8953f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:31.944605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6ec98-62d9-40ac-a24b-ad1e75d8953f None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:31.945110 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6ec98-62d9-40ac-a24b-ad1e75d8953f None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:31.947034 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6ec98-62d9-40ac-a24b-ad1e75d8953f None admin] PATH_INFO: `/v3/groups/91b7d5b4243043bd9dcb0b7289081f89` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:31.948712 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6ec98-62d9-40ac-a24b-ad1e75d8953f None admin] RBAC: Authorizing `identity:get_group(group_id=91b7d5b4243043bd9dcb0b7289081f89)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:31.951491 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6ec98-62d9-40ac-a24b-ad1e75d8953f None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:31.961216 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 197/411] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jun 27 15:34:31 2022] GET /identity/v3/groups/91b7d5b4243043bd9dcb0b7289081f89 => generated 226 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:31.966653 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6ec98-62d9-40ac-a24b-ad1e75d8953f None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:32.007615 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82b39d-0ddc-4a85-af73-eb473ef3ae11 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:32.045527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033ed2e-ddce-4f99-8319-df8e4f67e19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:32.046801 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033ed2e-ddce-4f99-8319-df8e4f67e19b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.047382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033ed2e-ddce-4f99-8319-df8e4f67e19b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.048055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033ed2e-ddce-4f99-8319-df8e4f67e19b None admin] PATH_INFO: `/v3/projects/53048925682f4806b3831f88058d3439` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.049514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033ed2e-ddce-4f99-8319-df8e4f67e19b None admin] RBAC: Authorizing `identity:get_project(project_id=53048925682f4806b3831f88058d3439)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:32.051318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033ed2e-ddce-4f99-8319-df8e4f67e19b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:32.053624 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 215/412] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:31 2022] GET /identity/v3/projects/53048925682f4806b3831f88058d3439 => generated 344 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:32.059147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033ed2e-ddce-4f99-8319-df8e4f67e19b None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:32.137120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6ec98-62d9-40ac-a24b-ad1e75d8953f None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:32.191972 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a1511-9524-4cf8-a4ab-79490c7acfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:32.195878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a1511-9524-4cf8-a4ab-79490c7acfb0 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.196963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a1511-9524-4cf8-a4ab-79490c7acfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.198004 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a1511-9524-4cf8-a4ab-79490c7acfb0 None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.201348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a1511-9524-4cf8-a4ab-79490c7acfb0 None admin] RBAC: Authorizing `identity:get_project(project_id=9974edfe18e14b8a9711b3f5c02808a9)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:32.204539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a1511-9524-4cf8-a4ab-79490c7acfb0 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:32.207478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 198/413] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:31 2022] GET /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:32.213248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a1511-9524-4cf8-a4ab-79490c7acfb0 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:32.253985 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:32.288246 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033ed2e-ddce-4f99-8319-df8e4f67e19b None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:32.298583 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:32.326457 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f71f8-de2b-450e-b22f-998291ec9631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:32.327745 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f71f8-de2b-450e-b22f-998291ec9631 None admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.328252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f71f8-de2b-450e-b22f-998291ec9631 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.328723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f71f8-de2b-450e-b22f-998291ec9631 None admin] PATH_INFO: `/v3/projects/53048925682f4806b3831f88058d3439/groups/e93c0b5734e0471d8abaf3d06e72a5b9/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.331028 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f71f8-de2b-450e-b22f-998291ec9631 None admin] RBAC: Authorizing `identity:create_grant(project_id=53048925682f4806b3831f88058d3439, group_id=e93c0b5734e0471d8abaf3d06e72a5b9, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:32.332731 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f71f8-de2b-450e-b22f-998291ec9631 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:32.350621 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 216/414] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Jun 27 15:34:32 2022] PUT /identity/v3/projects/53048925682f4806b3831f88058d3439/groups/e93c0b5734e0471d8abaf3d06e72a5b9/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:32.352297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a537c3-b03e-412d-a8a5-74a912d47e13 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.353370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a537c3-b03e-412d-a8a5-74a912d47e13 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.353904 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a537c3-b03e-412d-a8a5-74a912d47e13 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.361128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 217/415] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:32.363295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8c55d-1abf-45dd-9f78-3fe0e92a1ac8 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.364080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8c55d-1abf-45dd-9f78-3fe0e92a1ac8 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.364688 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8c55d-1abf-45dd-9f78-3fe0e92a1ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.390943 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-a2c8c55d-1abf-45dd-9f78-3fe0e92a1ac8 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:32.392076 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-a2c8c55d-1abf-45dd-9f78-3fe0e92a1ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:32.397416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a1511-9524-4cf8-a4ab-79490c7acfb0 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:32.445474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdabf5e-68eb-4e06-b8c0-866405193c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:32.447523 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdabf5e-68eb-4e06-b8c0-866405193c0d None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.448031 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdabf5e-68eb-4e06-b8c0-866405193c0d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.448965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdabf5e-68eb-4e06-b8c0-866405193c0d None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9/groups/91b7d5b4243043bd9dcb0b7289081f89/roles/478b9cd589e846588025a4b4ed2c07f0` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.452418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdabf5e-68eb-4e06-b8c0-866405193c0d None admin] RBAC: Authorizing `identity:create_grant(project_id=9974edfe18e14b8a9711b3f5c02808a9, group_id=91b7d5b4243043bd9dcb0b7289081f89, role_id=478b9cd589e846588025a4b4ed2c07f0)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:32.454356 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdabf5e-68eb-4e06-b8c0-866405193c0d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:32.472236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 199/416] 10.222.0.20 () {58 vars in 1442 bytes} [Mon Jun 27 15:34:32 2022] PUT /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9/groups/91b7d5b4243043bd9dcb0b7289081f89/roles/478b9cd589e846588025a4b4ed2c07f0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:34:32.529016 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 218/417] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:32.576923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdabf5e-68eb-4e06-b8c0-866405193c0d None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:32.686191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:32.706738 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdabf5e-68eb-4e06-b8c0-866405193c0d None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:32.746417 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703bf2e-8580-4823-9ae8-841f52fec5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:32.747317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703bf2e-8580-4823-9ae8-841f52fec5ec None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.747607 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703bf2e-8580-4823-9ae8-841f52fec5ec None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.747887 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703bf2e-8580-4823-9ae8-841f52fec5ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.759294 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703bf2e-8580-4823-9ae8-841f52fec5ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:32.767958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-b703bf2e-8580-4823-9ae8-841f52fec5ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:32.767958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:32.767958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:32.767958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:32.767958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:32.767958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:32.767958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:32.767958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:32.767958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:32.767958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:32.767958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:32.767958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:32.767958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:32.767958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:32.767958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:32.767958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:32.768908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:32.768908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:32.768908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:32.768908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:32.768908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:32.768908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:32.768908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:32.768908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:32.768908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:32.768908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:32.768908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:32.768908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:32.768908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:32.768908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:32.768908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:32.768908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:32.769715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:32.769715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:32.769715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:32.769715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:32.769715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:32.769715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:32.769715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:32.769715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:32.769715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:32.769715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:32.769715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:32.769715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:32.769715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 200/418] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:34:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:32.775886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8c55d-1abf-45dd-9f78-3fe0e92a1ac8 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:32.785395 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc66f75-544a-478b-9634-7e0febca0e87 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.785823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc66f75-544a-478b-9634-7e0febca0e87 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.786035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc66f75-544a-478b-9634-7e0febca0e87 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.787521 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 201/419] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:32.799727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8f027-6f4c-44b5-bb41-e4887bcf3f8a None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:32.799727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8f027-6f4c-44b5-bb41-e4887bcf3f8a None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:32.799727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8f027-6f4c-44b5-bb41-e4887bcf3f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:32.820127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-3ec8f027-6f4c-44b5-bb41-e4887bcf3f8a None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:32.822525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3ec8f027-6f4c-44b5-bb41-e4887bcf3f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:32.926483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 202/420] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:34:32.948259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:32.952137 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8c55d-1abf-45dd-9f78-3fe0e92a1ac8 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:32.966134 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8f027-6f4c-44b5-bb41-e4887bcf3f8a None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:33.007213 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fbaaf-0a72-477d-9952-29a4fc36d2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:33.009474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fbaaf-0a72-477d-9952-29a4fc36d2e1 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:33.011605 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fbaaf-0a72-477d-9952-29a4fc36d2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:33.012071 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fbaaf-0a72-477d-9952-29a4fc36d2e1 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:33.014062 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fbaaf-0a72-477d-9952-29a4fc36d2e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:33.015194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fbaaf-0a72-477d-9952-29a4fc36d2e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:33.016775 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fbaaf-0a72-477d-9952-29a4fc36d2e1 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:33.038769 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 219/421] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:33.045330 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fbaaf-0a72-477d-9952-29a4fc36d2e1 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:33.093305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:33.159563 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8f027-6f4c-44b5-bb41-e4887bcf3f8a None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:33.224981 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fbaaf-0a72-477d-9952-29a4fc36d2e1 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:33.227708 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e7525-aa82-4330-8227-1152eabe76d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:33.229121 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e7525-aa82-4330-8227-1152eabe76d3 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:33.229397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e7525-aa82-4330-8227-1152eabe76d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:33.229683 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59e7525-aa82-4330-8227-1152eabe76d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:33.252370 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59e7525-aa82-4330-8227-1152eabe76d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:33.260849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-e59e7525-aa82-4330-8227-1152eabe76d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:33.260849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:33.260849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:33.260849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:33.260849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:33.260849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:33.260849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:33.260849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:33.260849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:33.260849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:33.260849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:33.260849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:33.260849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:33.260849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:33.260849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:33.260849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:33.261808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:33.261808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:33.261808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:33.261808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:33.261808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:33.261808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:33.261808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:33.261808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:33.261808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:33.261808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:33.261808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:33.261808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:33.261808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:33.261808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:33.261808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:33.261808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:33.262619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:33.262619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:33.262619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:33.262619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:33.262619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:33.262619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:33.262619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:33.262619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:33.262619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:33.262619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:33.262619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:33.262619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:33.269705 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 203/422] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:33.276529 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e7525-aa82-4330-8227-1152eabe76d3 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:33.288466 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af351853-76f1-4eb1-a577-76f6428acdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:33.289124 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af351853-76f1-4eb1-a577-76f6428acdc3 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:33.289404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af351853-76f1-4eb1-a577-76f6428acdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:33.289694 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af351853-76f1-4eb1-a577-76f6428acdc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:33.323280 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:33.374178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af351853-76f1-4eb1-a577-76f6428acdc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:33.375642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af351853-76f1-4eb1-a577-76f6428acdc3 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:33.512864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 220/423] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:33.514580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af351853-76f1-4eb1-a577-76f6428acdc3 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:33.517856 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59e7525-aa82-4330-8227-1152eabe76d3 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:33.569949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:33.582679 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe0e53-7a89-4a47-863c-1efe4d3cad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:33.583386 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe0e53-7a89-4a47-863c-1efe4d3cad02 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:33.583636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe0e53-7a89-4a47-863c-1efe4d3cad02 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:33.583916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe0e53-7a89-4a47-863c-1efe4d3cad02 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:33.584675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe0e53-7a89-4a47-863c-1efe4d3cad02 None admin] RBAC: Adding query filter params (name=member) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:33.584985 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe0e53-7a89-4a47-863c-1efe4d3cad02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:33.586402 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe0e53-7a89-4a47-863c-1efe4d3cad02 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:33.604905 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 204/424] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:33.608877 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e5e82-2b49-4b87-b184-8a188b4b5d74 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:33.609164 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e5e82-2b49-4b87-b184-8a188b4b5d74 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:33.609443 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e5e82-2b49-4b87-b184-8a188b4b5d74 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:33.610858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 205/425] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:34:33.611681 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e5e82-2b49-4b87-b184-8a188b4b5d74 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:33.619938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:33.661840 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af351853-76f1-4eb1-a577-76f6428acdc3 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:33.704554 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba702164-6e44-4076-bc46-d7730a4603f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:33.705459 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba702164-6e44-4076-bc46-d7730a4603f7 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:33.705779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba702164-6e44-4076-bc46-d7730a4603f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:33.706063 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba702164-6e44-4076-bc46-d7730a4603f7 None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:33.707283 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba702164-6e44-4076-bc46-d7730a4603f7 None admin] RBAC: Authorizing `identity:get_project(project_id=c0f0739222694cedabb8f12e4ce82e9b)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:33.708843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba702164-6e44-4076-bc46-d7730a4603f7 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:33.711042 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 221/426] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:33 2022] GET /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:33.712969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af4908-fc90-48fd-a28d-8e38622b1832 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:33.713260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af4908-fc90-48fd-a28d-8e38622b1832 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:33.713660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af4908-fc90-48fd-a28d-8e38622b1832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:33.732771 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-89af4908-fc90-48fd-a28d-8e38622b1832 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:33.734026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-89af4908-fc90-48fd-a28d-8e38622b1832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:33.739562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e5e82-2b49-4b87-b184-8a188b4b5d74 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:33.778625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a5733-62ba-4cc0-a2d8-037095fda2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:33.779246 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a5733-62ba-4cc0-a2d8-037095fda2d6 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:33.779542 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a5733-62ba-4cc0-a2d8-037095fda2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:33.779853 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a5733-62ba-4cc0-a2d8-037095fda2d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:33.809203 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 222/427] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:33.810147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af4908-fc90-48fd-a28d-8e38622b1832 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:33.882083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:33.888765 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:33.902876 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a5733-62ba-4cc0-a2d8-037095fda2d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:33.904591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a5733-62ba-4cc0-a2d8-037095fda2d6 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:33.933067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af4908-fc90-48fd-a28d-8e38622b1832 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:33.970426 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638a008-9d65-4f8f-82b3-9bb57627641b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:33.971320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638a008-9d65-4f8f-82b3-9bb57627641b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:33.971607 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638a008-9d65-4f8f-82b3-9bb57627641b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:33.971884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638a008-9d65-4f8f-82b3-9bb57627641b None admin] PATH_INFO: `/v3/groups/91b7d5b4243043bd9dcb0b7289081f89` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:33.973330 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c638a008-9d65-4f8f-82b3-9bb57627641b None admin] RBAC: Authorizing `identity:get_group(group_id=91b7d5b4243043bd9dcb0b7289081f89)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:33.974741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c638a008-9d65-4f8f-82b3-9bb57627641b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:33.976728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 223/428] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jun 27 15:34:33 2022] GET /identity/v3/groups/91b7d5b4243043bd9dcb0b7289081f89 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:33.977537 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638a008-9d65-4f8f-82b3-9bb57627641b None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:34.015895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 206/429] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:34.017023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a5733-62ba-4cc0-a2d8-037095fda2d6 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:34.083995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638a008-9d65-4f8f-82b3-9bb57627641b None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:34.113041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:34.121011 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:34.124181 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c402510-6e88-4f2e-97d9-1ae1109fd301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:34.124987 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c402510-6e88-4f2e-97d9-1ae1109fd301 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.125272 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c402510-6e88-4f2e-97d9-1ae1109fd301 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.125720 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c402510-6e88-4f2e-97d9-1ae1109fd301 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.129713 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a5733-62ba-4cc0-a2d8-037095fda2d6 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:34.136689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c402510-6e88-4f2e-97d9-1ae1109fd301 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:34.144600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-4c402510-6e88-4f2e-97d9-1ae1109fd301 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:34.144600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:34.144600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:34.144600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:34.144600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:34.144600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:34.144600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:34.144600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:34.144600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:34.144600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:34.144600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:34.144600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:34.144600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:34.144600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:34.144600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:34.144600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:34.145460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:34.145460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:34.145460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:34.145460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:34.145460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:34.145460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:34.145460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:34.145460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:34.145460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:34.145460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:34.145460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:34.145460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:34.145460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:34.145460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:34.145460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:34.145460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:34.146254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:34.146254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:34.146254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:34.146254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:34.146254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:34.146254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:34.146254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:34.146254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:34.146254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:34.146254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:34.146254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 15:34:34.146254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:34.151810 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 224/430] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:34.152701 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c402510-6e88-4f2e-97d9-1ae1109fd301 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:34.172338 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50825733-d002-4a20-9b58-1ca92263057c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:34.173424 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50825733-d002-4a20-9b58-1ca92263057c None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.174009 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50825733-d002-4a20-9b58-1ca92263057c None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.174455 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50825733-d002-4a20-9b58-1ca92263057c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.175485 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50825733-d002-4a20-9b58-1ca92263057c None admin] RBAC: Adding query filter params (group.id=91b7d5b4243043bd9dcb0b7289081f89, role.id=478b9cd589e846588025a4b4ed2c07f0, scope.project.id=c0f0739222694cedabb8f12e4ce82e9b) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:34.175977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50825733-d002-4a20-9b58-1ca92263057c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:34.177787 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50825733-d002-4a20-9b58-1ca92263057c None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:34.185783 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 207/431] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jun 27 15:34:34 2022] GET /identity/v3/role_assignments?group.id=91b7d5b4243043bd9dcb0b7289081f89&scope.project.id=c0f0739222694cedabb8f12e4ce82e9b&role.id=478b9cd589e846588025a4b4ed2c07f0 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:34.186994 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50825733-d002-4a20-9b58-1ca92263057c None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:34.268060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c402510-6e88-4f2e-97d9-1ae1109fd301 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:34.306519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f65ce-b773-493c-abd1-b79ead4aa400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:34.307517 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f65ce-b773-493c-abd1-b79ead4aa400 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.307808 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f65ce-b773-493c-abd1-b79ead4aa400 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.308086 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f65ce-b773-493c-abd1-b79ead4aa400 None admin] PATH_INFO: `/v3/projects/c0f0739222694cedabb8f12e4ce82e9b` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.309325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f65ce-b773-493c-abd1-b79ead4aa400 None admin] RBAC: Authorizing `identity:get_project(project_id=c0f0739222694cedabb8f12e4ce82e9b)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:34.310837 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f65ce-b773-493c-abd1-b79ead4aa400 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:34.312145 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50825733-d002-4a20-9b58-1ca92263057c None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:34.312557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 225/432] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:34 2022] GET /identity/v3/projects/c0f0739222694cedabb8f12e4ce82e9b => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:34.318229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f65ce-b773-493c-abd1-b79ead4aa400 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:34.353915 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf623a-c78d-4087-b00a-7b5ab7a4afb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:34.354858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf623a-c78d-4087-b00a-7b5ab7a4afb4 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.355350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf623a-c78d-4087-b00a-7b5ab7a4afb4 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.355780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf623a-c78d-4087-b00a-7b5ab7a4afb4 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.356831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf623a-c78d-4087-b00a-7b5ab7a4afb4 None admin] RBAC: Adding query filter params (name=member) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:34.357300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf623a-c78d-4087-b00a-7b5ab7a4afb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:34.358911 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf623a-c78d-4087-b00a-7b5ab7a4afb4 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:34.369481 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 208/433] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:34:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:34.375374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf623a-c78d-4087-b00a-7b5ab7a4afb4 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:34.391905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:34.404153 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:34.443832 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f65ce-b773-493c-abd1-b79ead4aa400 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:34.481753 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce960c8-33dc-4887-81f7-2151b2c9b893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:34.483057 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce960c8-33dc-4887-81f7-2151b2c9b893 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.483550 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce960c8-33dc-4887-81f7-2151b2c9b893 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.484031 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce960c8-33dc-4887-81f7-2151b2c9b893 None admin] PATH_INFO: `/v3/groups/91b7d5b4243043bd9dcb0b7289081f89` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.485533 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce960c8-33dc-4887-81f7-2151b2c9b893 None admin] RBAC: Authorizing `identity:get_group(group_id=91b7d5b4243043bd9dcb0b7289081f89)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:34.487259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce960c8-33dc-4887-81f7-2151b2c9b893 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:34.489598 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 226/434] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jun 27 15:34:34 2022] GET /identity/v3/groups/91b7d5b4243043bd9dcb0b7289081f89 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:34.496019 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce960c8-33dc-4887-81f7-2151b2c9b893 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:34.496872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf623a-c78d-4087-b00a-7b5ab7a4afb4 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:34.540267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676bd3a-a4ea-4381-9645-9d4bb8ba0397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:34.541314 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676bd3a-a4ea-4381-9645-9d4bb8ba0397 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.541601 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676bd3a-a4ea-4381-9645-9d4bb8ba0397 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.541956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676bd3a-a4ea-4381-9645-9d4bb8ba0397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.606417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce960c8-33dc-4887-81f7-2151b2c9b893 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:34.619988 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676bd3a-a4ea-4381-9645-9d4bb8ba0397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:34.621571 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676bd3a-a4ea-4381-9645-9d4bb8ba0397 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:34.632798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:34.646077 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:34.648969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592125d-908a-486d-afeb-d92e62a40085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:34.650516 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592125d-908a-486d-afeb-d92e62a40085 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.651022 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592125d-908a-486d-afeb-d92e62a40085 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.651503 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592125d-908a-486d-afeb-d92e62a40085 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.653619 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592125d-908a-486d-afeb-d92e62a40085 None admin] RBAC: Adding query filter params (group.id=91b7d5b4243043bd9dcb0b7289081f89, role.id=5dc71eded4f64c08b2fb5f545b63b2f6, scope.project.id=c0f0739222694cedabb8f12e4ce82e9b) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:34.654400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592125d-908a-486d-afeb-d92e62a40085 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:34.656550 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592125d-908a-486d-afeb-d92e62a40085 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:34.669913 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 227/435] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jun 27 15:34:34 2022] GET /identity/v3/role_assignments?group.id=91b7d5b4243043bd9dcb0b7289081f89&scope.project.id=c0f0739222694cedabb8f12e4ce82e9b&role.id=5dc71eded4f64c08b2fb5f545b63b2f6 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:34.743190 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 209/436] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 15:34:34.749112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592125d-908a-486d-afeb-d92e62a40085 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:34.871562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d86dea-9362-456c-8b8e-fc46509ef298 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.871863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d86dea-9362-456c-8b8e-fc46509ef298 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.872049 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d86dea-9362-456c-8b8e-fc46509ef298 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.873489 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 210/437] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:34.878944 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c77e5-36bf-4652-8a4f-92862d760e59 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.879233 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c77e5-36bf-4652-8a4f-92862d760e59 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.879511 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c77e5-36bf-4652-8a4f-92862d760e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.885842 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592125d-908a-486d-afeb-d92e62a40085 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:34.903707 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-8d2c77e5-36bf-4652-8a4f-92862d760e59 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:34.903707 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-8d2c77e5-36bf-4652-8a4f-92862d760e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:34.926209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815044f4-944f-479f-b89d-36dc2e43ce1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:34.927188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815044f4-944f-479f-b89d-36dc2e43ce1f None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:34.927479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815044f4-944f-479f-b89d-36dc2e43ce1f None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:34.927758 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815044f4-944f-479f-b89d-36dc2e43ce1f None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:34.928973 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815044f4-944f-479f-b89d-36dc2e43ce1f None admin] RBAC: Authorizing `identity:get_project(project_id=9974edfe18e14b8a9711b3f5c02808a9)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:34.930458 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815044f4-944f-479f-b89d-36dc2e43ce1f None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:34.933420 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 228/438] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:34 2022] GET /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:34.938490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815044f4-944f-479f-b89d-36dc2e43ce1f None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:34.995739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 211/439] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:35.004813 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd46ac-be70-44e9-ab68-ade41c84f4cd None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:35.005108 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd46ac-be70-44e9-ab68-ade41c84f4cd None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:35.005398 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd46ac-be70-44e9-ab68-ade41c84f4cd None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:35.006948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 212/440] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:34:35.008880 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:35.012916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74582463-df2b-4520-9dd2-f66d8a5379fa None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:35.013327 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74582463-df2b-4520-9dd2-f66d8a5379fa None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:35.013738 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74582463-df2b-4520-9dd2-f66d8a5379fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:35.035345 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-74582463-df2b-4520-9dd2-f66d8a5379fa None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:35.036572 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-74582463-df2b-4520-9dd2-f66d8a5379fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:35.065312 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815044f4-944f-479f-b89d-36dc2e43ce1f None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:35.093797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:35.113628 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935015b-8564-4198-be12-89bc7b6db332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:35.115067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935015b-8564-4198-be12-89bc7b6db332 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:35.115290 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935015b-8564-4198-be12-89bc7b6db332 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:35.115579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935015b-8564-4198-be12-89bc7b6db332 None admin] PATH_INFO: `/v3/groups/91b7d5b4243043bd9dcb0b7289081f89` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:35.117084 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935015b-8564-4198-be12-89bc7b6db332 None admin] RBAC: Authorizing `identity:get_group(group_id=91b7d5b4243043bd9dcb0b7289081f89)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:35.119345 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935015b-8564-4198-be12-89bc7b6db332 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:35.120621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 213/441] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:34:35.120851 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 229/442] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jun 27 15:34:34 2022] GET /identity/v3/groups/91b7d5b4243043bd9dcb0b7289081f89 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:35.121453 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74582463-df2b-4520-9dd2-f66d8a5379fa None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:35.127571 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935015b-8564-4198-be12-89bc7b6db332 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:35.247672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935015b-8564-4198-be12-89bc7b6db332 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:35.249035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74582463-df2b-4520-9dd2-f66d8a5379fa None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:35.276346 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:35.292226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6ed34-7517-41ef-b155-e178a745eb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:35.292889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6ed34-7517-41ef-b155-e178a745eb74 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:35.293174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6ed34-7517-41ef-b155-e178a745eb74 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:35.293453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6ed34-7517-41ef-b155-e178a745eb74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:35.294060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b05036-d5d4-496e-8bdd-3c2ea808e3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:35.294343 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6ed34-7517-41ef-b155-e178a745eb74 None admin] RBAC: Adding query filter params (group.id=91b7d5b4243043bd9dcb0b7289081f89, role.id=5dc71eded4f64c08b2fb5f545b63b2f6, scope.project.id=9974edfe18e14b8a9711b3f5c02808a9) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:35.294660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6ed34-7517-41ef-b155-e178a745eb74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:35.294887 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b05036-d5d4-496e-8bdd-3c2ea808e3c0 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:35.295237 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b05036-d5d4-496e-8bdd-3c2ea808e3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:35.296740 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b05036-d5d4-496e-8bdd-3c2ea808e3c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:35.296926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6ed34-7517-41ef-b155-e178a745eb74 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:35.302921 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 230/443] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jun 27 15:34:35 2022] GET /identity/v3/role_assignments?group.id=91b7d5b4243043bd9dcb0b7289081f89&scope.project.id=9974edfe18e14b8a9711b3f5c02808a9&role.id=5dc71eded4f64c08b2fb5f545b63b2f6 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:35.303588 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6ed34-7517-41ef-b155-e178a745eb74 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:35.311978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b05036-d5d4-496e-8bdd-3c2ea808e3c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:35.322900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-23b05036-d5d4-496e-8bdd-3c2ea808e3c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:35.322900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:35.322900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:35.322900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:35.322900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:35.322900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:35.322900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:35.322900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:35.322900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:35.322900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:35.322900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:35.322900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:35.322900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:35.322900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:35.322900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:35.322900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:35.323894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:35.323894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:35.323894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:35.323894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:35.323894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:35.323894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:35.323894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:35.323894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:35.323894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:35.323894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:35.323894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:35.323894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:35.323894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:35.323894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:35.323894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:35.323894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:35.324669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:35.324669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:35.324669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:35.324669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:35.324669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:35.324669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:35.324669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:35.324669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:35.324669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:35.324669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:35.324669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:35.324669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:35.324669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 214/444] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:34:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:35.330382 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b05036-d5d4-496e-8bdd-3c2ea808e3c0 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:35.385599 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:35.429447 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6ed34-7517-41ef-b155-e178a745eb74 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:35.454254 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b05036-d5d4-496e-8bdd-3c2ea808e3c0 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:35.475058 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171297c6-c305-4ad5-ab89-ebbb630cf293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:35.475312 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171297c6-c305-4ad5-ab89-ebbb630cf293 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:35.475600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171297c6-c305-4ad5-ab89-ebbb630cf293 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:35.475861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171297c6-c305-4ad5-ab89-ebbb630cf293 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:35.488317 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171297c6-c305-4ad5-ab89-ebbb630cf293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:35.496553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-171297c6-c305-4ad5-ab89-ebbb630cf293 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:34:35.496553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:35.496553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:35.496553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:35.496553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:35.496553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:35.496553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:35.496553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:35.496553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:35.496553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:35.496553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:35.496553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:35.496553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:35.496553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:35.496553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:35.496553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:35.497523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:35.497523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:35.497523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:35.497523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:35.497523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:35.497523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:35.497523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:35.497523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:35.497523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:35.497523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:35.497523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:35.497523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:35.497523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:35.497523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:35.497523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:35.497523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:35.498364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:35.498364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:35.498364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:35.498364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:35.498364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:35.498364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:35.498364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:35.498364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:35.498364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:35.498364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:35.498364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:34:35.498364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:35.498364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 231/445] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:34:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:35.503948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36faf8-d998-4e31-b6bd-6dd37185a971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:35.504179 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171297c6-c305-4ad5-ab89-ebbb630cf293 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:35.504634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36faf8-d998-4e31-b6bd-6dd37185a971 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:35.504920 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36faf8-d998-4e31-b6bd-6dd37185a971 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:35.505276 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36faf8-d998-4e31-b6bd-6dd37185a971 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:35.506591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36faf8-d998-4e31-b6bd-6dd37185a971 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:35.506591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36faf8-d998-4e31-b6bd-6dd37185a971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:35.507732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36faf8-d998-4e31-b6bd-6dd37185a971 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:35.518440 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 215/446] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:35.523944 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36faf8-d998-4e31-b6bd-6dd37185a971 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:35.567037 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:35.637078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171297c6-c305-4ad5-ab89-ebbb630cf293 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:35.644469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36faf8-d998-4e31-b6bd-6dd37185a971 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:35.646179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:35.687257 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9cd55-51b5-4c3b-bd49-d487040c3de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:35.687980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9cd55-51b5-4c3b-bd49-d487040c3de8 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:35.688355 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9cd55-51b5-4c3b-bd49-d487040c3de8 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:35.688531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9cd55-51b5-4c3b-bd49-d487040c3de8 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:35.689715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9cd55-51b5-4c3b-bd49-d487040c3de8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:35.689715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9cd55-51b5-4c3b-bd49-d487040c3de8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:35.691336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9cd55-51b5-4c3b-bd49-d487040c3de8 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:35.693369 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5618bb-af12-4ae7-8cf4-fe2653dbc30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:35.694255 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5618bb-af12-4ae7-8cf4-fe2653dbc30d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:35.694546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5618bb-af12-4ae7-8cf4-fe2653dbc30d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:35.694872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5618bb-af12-4ae7-8cf4-fe2653dbc30d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:35.703120 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 232/447] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:35.711522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9cd55-51b5-4c3b-bd49-d487040c3de8 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:35.775155 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5618bb-af12-4ae7-8cf4-fe2653dbc30d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:35.776665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5618bb-af12-4ae7-8cf4-fe2653dbc30d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:35.813037 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9cd55-51b5-4c3b-bd49-d487040c3de8 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:35.825253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:35.848088 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb80ab-cd0d-4d5b-8371-03fce91ef9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:35.848756 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb80ab-cd0d-4d5b-8371-03fce91ef9c0 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:35.849035 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb80ab-cd0d-4d5b-8371-03fce91ef9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:35.849307 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb80ab-cd0d-4d5b-8371-03fce91ef9c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:35.878778 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 216/448] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 15:34:35.884832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5618bb-af12-4ae7-8cf4-fe2653dbc30d None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:35.887191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:35.917844 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb80ab-cd0d-4d5b-8371-03fce91ef9c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:35.919303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb80ab-cd0d-4d5b-8371-03fce91ef9c0 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:35.984254 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5618bb-af12-4ae7-8cf4-fe2653dbc30d None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:36.016374 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 233/449] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 15:34:36.017926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba73b6-4b7a-4fda-875a-d61b9cfeee97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:36.018796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba73b6-4b7a-4fda-875a-d61b9cfeee97 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:36.019078 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba73b6-4b7a-4fda-875a-d61b9cfeee97 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:36.019351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba73b6-4b7a-4fda-875a-d61b9cfeee97 None admin] PATH_INFO: `/v3/projects/9974edfe18e14b8a9711b3f5c02808a9` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:36.020538 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba73b6-4b7a-4fda-875a-d61b9cfeee97 None admin] RBAC: Authorizing `identity:get_project(project_id=9974edfe18e14b8a9711b3f5c02808a9)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:36.021892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb80ab-cd0d-4d5b-8371-03fce91ef9c0 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:36.022017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba73b6-4b7a-4fda-875a-d61b9cfeee97 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:36.023917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 217/450] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:35 2022] GET /identity/v3/projects/9974edfe18e14b8a9711b3f5c02808a9 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:36.029567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba73b6-4b7a-4fda-875a-d61b9cfeee97 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:36.032550 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:36.098984 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:36.129365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb80ab-cd0d-4d5b-8371-03fce91ef9c0 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:36.131087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba73b6-4b7a-4fda-875a-d61b9cfeee97 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:36.164497 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412a39a-cdde-4fe1-9b82-aeff24c8846e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:36.165890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412a39a-cdde-4fe1-9b82-aeff24c8846e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:36.166178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412a39a-cdde-4fe1-9b82-aeff24c8846e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:36.166457 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412a39a-cdde-4fe1-9b82-aeff24c8846e None admin] PATH_INFO: `/v3/projects/53048925682f4806b3831f88058d3439` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:36.166807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1cdcf-0bbf-4768-b482-c286912df231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:36.167655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2412a39a-cdde-4fe1-9b82-aeff24c8846e None admin] RBAC: Authorizing `identity:get_project(project_id=53048925682f4806b3831f88058d3439)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:36.167744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1cdcf-0bbf-4768-b482-c286912df231 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:36.167936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1cdcf-0bbf-4768-b482-c286912df231 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:36.168212 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1cdcf-0bbf-4768-b482-c286912df231 None admin] PATH_INFO: `/v3/groups/91b7d5b4243043bd9dcb0b7289081f89` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:36.169087 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2412a39a-cdde-4fe1-9b82-aeff24c8846e None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:36.169382 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1cdcf-0bbf-4768-b482-c286912df231 None admin] RBAC: Authorizing `identity:get_group(group_id=91b7d5b4243043bd9dcb0b7289081f89)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:36.170833 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 234/451] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:34:36 2022] GET /identity/v3/projects/53048925682f4806b3831f88058d3439 => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:36.171011 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1cdcf-0bbf-4768-b482-c286912df231 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:36.172441 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 218/452] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jun 27 15:34:36 2022] GET /identity/v3/groups/91b7d5b4243043bd9dcb0b7289081f89 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:36.176277 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412a39a-cdde-4fe1-9b82-aeff24c8846e None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:36.177871 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1cdcf-0bbf-4768-b482-c286912df231 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:36.251221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:36.275125 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412a39a-cdde-4fe1-9b82-aeff24c8846e None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:36.278435 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1cdcf-0bbf-4768-b482-c286912df231 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:36.309098 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467bef4-dc9c-427d-8f68-3804ea1c2926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:36.310073 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467bef4-dc9c-427d-8f68-3804ea1c2926 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:36.310362 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467bef4-dc9c-427d-8f68-3804ea1c2926 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:36.310634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467bef4-dc9c-427d-8f68-3804ea1c2926 None admin] PATH_INFO: `/v3/groups/e93c0b5734e0471d8abaf3d06e72a5b9` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:36.311806 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467bef4-dc9c-427d-8f68-3804ea1c2926 None admin] RBAC: Authorizing `identity:get_group(group_id=e93c0b5734e0471d8abaf3d06e72a5b9)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:36.312630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edf46a-bdfd-4e0c-934c-949f09acf847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:36.313228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467bef4-dc9c-427d-8f68-3804ea1c2926 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:36.313319 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edf46a-bdfd-4e0c-934c-949f09acf847 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:36.313546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edf46a-bdfd-4e0c-934c-949f09acf847 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:36.313863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edf46a-bdfd-4e0c-934c-949f09acf847 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:36.314701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8edf46a-bdfd-4e0c-934c-949f09acf847 None admin] RBAC: Adding query filter params (group.id=91b7d5b4243043bd9dcb0b7289081f89, role.id=478b9cd589e846588025a4b4ed2c07f0, scope.project.id=9974edfe18e14b8a9711b3f5c02808a9) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:36.315020 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8edf46a-bdfd-4e0c-934c-949f09acf847 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:36.315229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 235/453] 10.222.0.20 () {56 vars in 1180 bytes} [Mon Jun 27 15:34:36 2022] GET /identity/v3/groups/e93c0b5734e0471d8abaf3d06e72a5b9 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:36.316438 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8edf46a-bdfd-4e0c-934c-949f09acf847 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:36.320500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467bef4-dc9c-427d-8f68-3804ea1c2926 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:36.324220 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 219/454] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jun 27 15:34:36 2022] GET /identity/v3/role_assignments?group.id=91b7d5b4243043bd9dcb0b7289081f89&scope.project.id=9974edfe18e14b8a9711b3f5c02808a9&role.id=478b9cd589e846588025a4b4ed2c07f0 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:36.429188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467bef4-dc9c-427d-8f68-3804ea1c2926 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:36.466354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f27bf1-6803-4c54-b709-4fc1f673aece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:36.467043 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f27bf1-6803-4c54-b709-4fc1f673aece None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:36.467332 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f27bf1-6803-4c54-b709-4fc1f673aece None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:36.467608 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f27bf1-6803-4c54-b709-4fc1f673aece None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:36.468457 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f27bf1-6803-4c54-b709-4fc1f673aece None admin] RBAC: Adding query filter params (group.id=e93c0b5734e0471d8abaf3d06e72a5b9, role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.project.id=53048925682f4806b3831f88058d3439) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:36.468773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f27bf1-6803-4c54-b709-4fc1f673aece None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:36.470252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f27bf1-6803-4c54-b709-4fc1f673aece None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:36.476593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 236/455] 10.222.0.20 () {56 vars in 1376 bytes} [Mon Jun 27 15:34:36 2022] GET /identity/v3/role_assignments?group.id=e93c0b5734e0471d8abaf3d06e72a5b9&scope.project.id=53048925682f4806b3831f88058d3439&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:39.066998 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee642e-a5fe-4015-a623-e8a92b1bdd47 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.067797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee642e-a5fe-4015-a623-e8a92b1bdd47 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.067930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee642e-a5fe-4015-a623-e8a92b1bdd47 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.070746 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 220/456] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:39.079376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b8b46-0672-4b53-8132-72d9739391c6 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.080059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b8b46-0672-4b53-8132-72d9739391c6 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.080341 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b8b46-0672-4b53-8132-72d9739391c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.121557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-0a2b8b46-0672-4b53-8132-72d9739391c6 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:39.123523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0a2b8b46-0672-4b53-8132-72d9739391c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:39.126981 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:39.269737 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 237/457] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:34:39.351126 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee642e-a5fe-4015-a623-e8a92b1bdd47 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.356968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:39.516586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee642e-a5fe-4015-a623-e8a92b1bdd47 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:39.561375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e3f36-3e74-4c1c-b3ed-faae6fae055c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:39.562219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e3f36-3e74-4c1c-b3ed-faae6fae055c None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.562502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e3f36-3e74-4c1c-b3ed-faae6fae055c None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.562763 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e3f36-3e74-4c1c-b3ed-faae6fae055c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.575511 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e3f36-3e74-4c1c-b3ed-faae6fae055c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:39.577025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e3f36-3e74-4c1c-b3ed-faae6fae055c None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:39.591489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba0003-23e2-4a02-a479-db43d252ad41 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.591967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba0003-23e2-4a02-a479-db43d252ad41 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.592462 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba0003-23e2-4a02-a479-db43d252ad41 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.595344 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 238/458] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:39.601956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-c64e3f36-3e74-4c1c-b3ed-faae6fae055c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:39.601956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:39.601956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:39.601956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:39.601956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:39.601956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:39.601956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:39.601956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:39.601956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:39.601956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:39.601956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:39.601956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:39.601956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:39.601956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:39.601956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:39.605086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:39.605086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:39.605086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:39.605086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:39.605086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:39.605086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:39.605086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:39.605086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:39.605086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:39.605086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:39.605086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:39.605086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:39.605086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:39.605086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:39.605086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:39.605086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:39.605086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:39.606872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:39.606872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:39.606872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:39.606872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:39.606872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 221/459] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:39.607397 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6c1c1-de0f-426f-92e2-0ab44cd6d7d6 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.607957 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6c1c1-de0f-426f-92e2-0ab44cd6d7d6 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.613663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6c1c1-de0f-426f-92e2-0ab44cd6d7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.617471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e3f36-3e74-4c1c-b3ed-faae6fae055c None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.658036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-83b6c1c1-de0f-426f-92e2-0ab44cd6d7d6 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:39.661221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-83b6c1c1-de0f-426f-92e2-0ab44cd6d7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:39.678828 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:39.726633 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e3f36-3e74-4c1c-b3ed-faae6fae055c None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:39.764225 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2bd6b-1201-4f24-860b-654be5689f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:39.765980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2bd6b-1201-4f24-860b-654be5689f33 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.766787 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2bd6b-1201-4f24-860b-654be5689f33 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.767547 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2bd6b-1201-4f24-860b-654be5689f33 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.769325 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2bd6b-1201-4f24-860b-654be5689f33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:39.770174 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2bd6b-1201-4f24-860b-654be5689f33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:39.773430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2bd6b-1201-4f24-860b-654be5689f33 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:39.788839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 239/460] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:39.790286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4fa2e-5740-4c79-9cf0-18aaf7f17a9c None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.790578 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4fa2e-5740-4c79-9cf0-18aaf7f17a9c None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.790886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4fa2e-5740-4c79-9cf0-18aaf7f17a9c None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.797666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 240/461] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:39.812638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08a92e-9551-4002-8808-220895d52c26 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.812901 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08a92e-9551-4002-8808-220895d52c26 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.813195 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08a92e-9551-4002-8808-220895d52c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:39.818790 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 222/462] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:39.820813 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2bd6b-1201-4f24-860b-654be5689f33 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.840634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-0f08a92e-9551-4002-8808-220895d52c26 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:39.841871 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0f08a92e-9551-4002-8808-220895d52c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:39.912012 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:39.919159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 241/463] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:34:39.919767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08a92e-9551-4002-8808-220895d52c26 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:39.946704 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2bd6b-1201-4f24-860b-654be5689f33 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:39.964463 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:39.987221 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7121c-02eb-4d49-a32c-e613d743e9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:39.988020 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7121c-02eb-4d49-a32c-e613d743e9df None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:39.988308 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7121c-02eb-4d49-a32c-e613d743e9df None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:39.988590 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7121c-02eb-4d49-a32c-e613d743e9df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.000177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7121c-02eb-4d49-a32c-e613d743e9df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:40.001793 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7121c-02eb-4d49-a32c-e613d743e9df None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.021508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-31e7121c-02eb-4d49-a32c-e613d743e9df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:40.021508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:40.021508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:40.021508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:40.021508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:40.021508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:40.021508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:40.021508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:40.021508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:40.021508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:40.021508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:40.021508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:40.021508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:40.021508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:40.021508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:40.022641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:40.022641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:40.022641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:40.022641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:40.022641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:40.022641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:40.022641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:40.022641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:40.022641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:40.022641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:40.022641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:40.022641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:40.022641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:40.022641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:40.022641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:40.022641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:40.022641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:40.023512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:40.023512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:40.023512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:40.023512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:40.023512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 223/464] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:40.024111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db591dbb-d059-4d7c-a86b-e5b263d31b05 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.024392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db591dbb-d059-4d7c-a86b-e5b263d31b05 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.024703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db591dbb-d059-4d7c-a86b-e5b263d31b05 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.029759 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 224/465] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:40.029759 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db591dbb-d059-4d7c-a86b-e5b263d31b05 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:40.030315 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08a92e-9551-4002-8808-220895d52c26 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:40.066533 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02264f5a-99d1-4f4e-83c0-18f4421ed9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:40.067259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02264f5a-99d1-4f4e-83c0-18f4421ed9ca None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.067521 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02264f5a-99d1-4f4e-83c0-18f4421ed9ca None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.067801 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02264f5a-99d1-4f4e-83c0-18f4421ed9ca None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.068510 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02264f5a-99d1-4f4e-83c0-18f4421ed9ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:40.069975 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02264f5a-99d1-4f4e-83c0-18f4421ed9ca None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.101680 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 242/466] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jun 27 15:34:39 2022] POST /identity/v3/users => generated 246 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:40.101680 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02264f5a-99d1-4f4e-83c0-18f4421ed9ca None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:40.168430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db591dbb-d059-4d7c-a86b-e5b263d31b05 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:40.201293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:40.217196 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2bc428-1f54-4a6c-b69c-f8d863af8c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:40.218838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bc428-1f54-4a6c-b69c-f8d863af8c76 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.219500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bc428-1f54-4a6c-b69c-f8d863af8c76 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.219855 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bc428-1f54-4a6c-b69c-f8d863af8c76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.235243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2bc428-1f54-4a6c-b69c-f8d863af8c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:40.237465 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2bc428-1f54-4a6c-b69c-f8d863af8c76 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.258630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02264f5a-99d1-4f4e-83c0-18f4421ed9ca None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:40.264901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-6d2bc428-1f54-4a6c-b69c-f8d863af8c76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:40.264901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:40.264901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:40.264901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:40.264901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:40.264901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:40.264901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:40.264901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:40.264901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:40.264901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:40.264901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:40.264901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:40.264901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:40.264901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:40.264901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:40.265795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:40.265795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:40.265795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:40.265795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:40.265795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:40.265795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:40.265795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:40.265795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:40.265795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:40.265795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:40.265795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:40.265795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:40.265795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:40.265795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:40.265795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:40.265795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:40.265795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:40.266640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:40.266640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:40.266640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:40.266640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:40.266640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 225/467] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:40.269954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77266693-2d15-4557-9aa9-1b0941c515b4 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.269954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77266693-2d15-4557-9aa9-1b0941c515b4 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.269954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77266693-2d15-4557-9aa9-1b0941c515b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.291407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-77266693-2d15-4557-9aa9-1b0941c515b4 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:40.292816 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-77266693-2d15-4557-9aa9-1b0941c515b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:40.309089 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b2631-f15b-42a2-b030-cb46300551d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:40.309911 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b2631-f15b-42a2-b030-cb46300551d7 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.310105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:40.310259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b2631-f15b-42a2-b030-cb46300551d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.310659 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b2631-f15b-42a2-b030-cb46300551d7 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.311409 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b2631-f15b-42a2-b030-cb46300551d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:40.311722 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b2631-f15b-42a2-b030-cb46300551d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:40.313497 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b2631-f15b-42a2-b030-cb46300551d7 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.336467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 243/468] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:40.337344 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b2631-f15b-42a2-b030-cb46300551d7 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:40.390867 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 226/469] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:40.391979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77266693-2d15-4557-9aa9-1b0941c515b4 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:40.467134 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b2631-f15b-42a2-b030-cb46300551d7 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:40.483661 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:40.507804 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358e219-1d86-4aef-b930-55d26e7e3362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:40.508844 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358e219-1d86-4aef-b930-55d26e7e3362 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.509974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358e219-1d86-4aef-b930-55d26e7e3362 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.510456 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358e219-1d86-4aef-b930-55d26e7e3362 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.511407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358e219-1d86-4aef-b930-55d26e7e3362 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:40.511934 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358e219-1d86-4aef-b930-55d26e7e3362 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:40.513508 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358e219-1d86-4aef-b930-55d26e7e3362 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.522192 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77266693-2d15-4557-9aa9-1b0941c515b4 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:40.527009 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 244/470] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:40.528110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358e219-1d86-4aef-b930-55d26e7e3362 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:40.567181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786709e6-9ccd-4289-adc9-42e691b8918d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:40.568234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786709e6-9ccd-4289-adc9-42e691b8918d None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.568763 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786709e6-9ccd-4289-adc9-42e691b8918d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.569233 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786709e6-9ccd-4289-adc9-42e691b8918d None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.570224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786709e6-9ccd-4289-adc9-42e691b8918d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:40.572602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786709e6-9ccd-4289-adc9-42e691b8918d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.596319 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 227/471] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jun 27 15:34:40 2022] POST /identity/v3/users => generated 248 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:40.597602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786709e6-9ccd-4289-adc9-42e691b8918d None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:40.619996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:40.714934 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358e219-1d86-4aef-b930-55d26e7e3362 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:40.754967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ab774-f9cb-410a-9c56-dac4475ad91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:40.756150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ab774-f9cb-410a-9c56-dac4475ad91a None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.756664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ab774-f9cb-410a-9c56-dac4475ad91a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.757141 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ab774-f9cb-410a-9c56-dac4475ad91a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.770252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ab774-f9cb-410a-9c56-dac4475ad91a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:40.772088 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ab774-f9cb-410a-9c56-dac4475ad91a None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.776714 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786709e6-9ccd-4289-adc9-42e691b8918d None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:40.794897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-fb6ab774-f9cb-410a-9c56-dac4475ad91a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:40.794897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:40.794897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:40.794897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:40.794897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:40.794897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:40.794897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:40.794897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:40.794897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:40.794897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:40.794897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:40.794897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:40.794897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:40.794897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:40.794897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:40.795843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:40.795843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:40.795843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:40.795843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:40.795843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:40.795843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:40.795843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:40.795843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:40.795843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:40.795843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:40.795843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:40.795843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:40.795843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:40.795843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:40.795843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:40.795843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:40.795843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:40.797349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:40.797349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:40.797349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:40.797349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:40.797349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 245/472] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:40.797910 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f0ddf-f3e2-4a89-92e9-68e7a042df91 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.798204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f0ddf-f3e2-4a89-92e9-68e7a042df91 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.798478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f0ddf-f3e2-4a89-92e9-68e7a042df91 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.800060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 246/473] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:40.801949 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f0ddf-f3e2-4a89-92e9-68e7a042df91 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:40.825041 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aceded-932d-4394-916c-47b612b581f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:40.826088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aceded-932d-4394-916c-47b612b581f3 None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.827541 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aceded-932d-4394-916c-47b612b581f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.828159 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aceded-932d-4394-916c-47b612b581f3 None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.829096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aceded-932d-4394-916c-47b612b581f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:40.830663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aceded-932d-4394-916c-47b612b581f3 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:40.845938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:40.855490 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 228/474] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jun 27 15:34:40 2022] POST /identity/v3/users => generated 248 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:40.857017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23354e4-dab6-46c7-8f7b-2ede7ba7204b None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:40.857473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23354e4-dab6-46c7-8f7b-2ede7ba7204b None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:40.857978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23354e4-dab6-46c7-8f7b-2ede7ba7204b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:40.891643 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-c23354e4-dab6-46c7-8f7b-2ede7ba7204b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:40.891643 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-c23354e4-dab6-46c7-8f7b-2ede7ba7204b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:40.971923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f0ddf-f3e2-4a89-92e9-68e7a042df91 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:40.982456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:41.002135 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 229/475] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:41.021621 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874389cc-a534-4525-b20f-247af6a2d29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:41.022283 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874389cc-a534-4525-b20f-247af6a2d29c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:41.022572 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874389cc-a534-4525-b20f-247af6a2d29c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:41.022848 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874389cc-a534-4525-b20f-247af6a2d29c None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:41.023664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874389cc-a534-4525-b20f-247af6a2d29c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:41.023972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874389cc-a534-4525-b20f-247af6a2d29c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:41.026546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874389cc-a534-4525-b20f-247af6a2d29c None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:41.037105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23354e4-dab6-46c7-8f7b-2ede7ba7204b None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:41.053159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 247/476] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:41.059434 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874389cc-a534-4525-b20f-247af6a2d29c None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:41.168314 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23354e4-dab6-46c7-8f7b-2ede7ba7204b None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:41.173591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:41.195919 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874389cc-a534-4525-b20f-247af6a2d29c None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:41.207089 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db689b9-9f3e-47c8-a4b1-3b459d772416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:41.207878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db689b9-9f3e-47c8-a4b1-3b459d772416 None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:41.208230 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db689b9-9f3e-47c8-a4b1-3b459d772416 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:41.208575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db689b9-9f3e-47c8-a4b1-3b459d772416 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:41.209394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db689b9-9f3e-47c8-a4b1-3b459d772416 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:41.210930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db689b9-9f3e-47c8-a4b1-3b459d772416 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:41.218997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.sql.core [None req-0db689b9-9f3e-47c8-a4b1-3b459d772416 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 27 15:34:41.218997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 27 15:34:41.218997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [parameters: {'id': 'dda640b819014474a752846773b6af7a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 27 15:34:41.218997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91941) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 27 15:34:41.239455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35138807-c7fc-4734-b977-c1338232a9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:41.240388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35138807-c7fc-4734-b977-c1338232a9c9 None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:41.240502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-0db689b9-9f3e-47c8-a4b1-3b459d772416 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 27 15:34:41.240502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:41.240502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 15:34:41.240502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 15:34:41.240502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 15:34:41.240502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 15:34:41.240502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 15:34:41.240502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 15:34:41.240502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 15:34:41.240502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application conn.query(q) Jun 27 15:34:41.240502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 15:34:41.240502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 15:34:41.240502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 15:34:41.240502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application result.read() Jun 27 15:34:41.240502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 15:34:41.241439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 15:34:41.241439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 15:34:41.241439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 15:34:41.241439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 15:34:41.241439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 15:34:41.241439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 15:34:41.241439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 15:34:41.241439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 27 15:34:41.241439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:41.241439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 27 15:34:41.241439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:41.241439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:41.241439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 27 15:34:41.241439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 27 15:34:41.241439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 27 15:34:41.241439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return ref.to_dict() Jun 27 15:34:41.241439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 15:34:41.241439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application next(self.gen) Jun 27 15:34:41.242394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 27 15:34:41.242394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application yield resource Jun 27 15:34:41.242394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 15:34:41.242394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application next(self.gen) Jun 27 15:34:41.242394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 27 15:34:41.242394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self.session.rollback() Jun 27 15:34:41.242394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 27 15:34:41.242394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self.force_reraise() Jun 27 15:34:41.242394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 27 15:34:41.242394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise self.value Jun 27 15:34:41.242394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 27 15:34:41.242394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 27 15:34:41.242394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 27 15:34:41.242394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application session.commit() Jun 27 15:34:41.242394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 27 15:34:41.242394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 27 15:34:41.242394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 27 15:34:41.243243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._prepare_impl() Jun 27 15:34:41.243243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 27 15:34:41.243243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self.session.flush() Jun 27 15:34:41.243243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 27 15:34:41.243243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._flush(objects) Jun 27 15:34:41.243243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 27 15:34:41.243243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 27 15:34:41.243243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 27 15:34:41.243243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application compat.raise_( Jun 27 15:34:41.243243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 15:34:41.243243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception Jun 27 15:34:41.243243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 27 15:34:41.243243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application flush_context.execute() Jun 27 15:34:41.243243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 27 15:34:41.243243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rec.execute(self) Jun 27 15:34:41.243243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 27 15:34:41.243243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 27 15:34:41.244167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 27 15:34:41.244167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 27 15:34:41.244167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 27 15:34:41.244167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 27 15:34:41.244167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 27 15:34:41.244167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 27 15:34:41.244167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 27 15:34:41.244167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 27 15:34:41.244167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 27 15:34:41.244167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 27 15:34:41.244167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 27 15:34:41.244167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 27 15:34:41.244167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 27 15:34:41.244167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 27 15:34:41.244167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 15:34:41.244167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception Jun 27 15:34:41.244990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 15:34:41.244990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 15:34:41.244990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 15:34:41.244990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 15:34:41.244990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 15:34:41.244990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 15:34:41.244990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 15:34:41.244990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application conn.query(q) Jun 27 15:34:41.244990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 15:34:41.244990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 15:34:41.244990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 15:34:41.244990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application result.read() Jun 27 15:34:41.244990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 15:34:41.244990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 15:34:41.244990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 15:34:41.244990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 15:34:41.246562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 15:34:41.246562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 15:34:41.246562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 15:34:41.246562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 15:34:41.246562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 27 15:34:41.246562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 27 15:34:41.246562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [parameters: {'id': 'dda640b819014474a752846773b6af7a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 27 15:34:41.246562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 27 15:34:41.246562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:41.246562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:34:41.246562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:41.246562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:41.246562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:41.246562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:41.246562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:41.246562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:41.248171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:41.248171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:41.248171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:41.248171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:41.248171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:41.248171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:41.248171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 27 15:34:41.248171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 27 15:34:41.248171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:41.248171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:41.248171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 27 15:34:41.248171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 27 15:34:41.248171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 27 15:34:41.248171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 27 15:34:41.248171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 27 15:34:41.248171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:41.250187 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 230/477] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Jun 27 15:34:41 2022] POST /identity/v3/roles => generated 144 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:34:41.250187 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db689b9-9f3e-47c8-a4b1-3b459d772416 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:41.251915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35138807-c7fc-4734-b977-c1338232a9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:41.251915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35138807-c7fc-4734-b977-c1338232a9c9 None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:41.252338 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35138807-c7fc-4734-b977-c1338232a9c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:41.255264 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35138807-c7fc-4734-b977-c1338232a9c9 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:41.280281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 248/478] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jun 27 15:34:41 2022] POST /identity/v3/users => generated 249 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:41.333408 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:41.388634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db689b9-9f3e-47c8-a4b1-3b459d772416 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:41.428357 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2825489-7caa-4f47-9bdf-e5e7c9138a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:41.429586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2825489-7caa-4f47-9bdf-e5e7c9138a2e None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:41.430037 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2825489-7caa-4f47-9bdf-e5e7c9138a2e None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:41.430462 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2825489-7caa-4f47-9bdf-e5e7c9138a2e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:41.442589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2825489-7caa-4f47-9bdf-e5e7c9138a2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:41.451161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-b2825489-7caa-4f47-9bdf-e5e7c9138a2e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:41.451161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:41.451161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:41.451161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:41.451161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:41.451161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:41.451161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:41.451161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:41.451161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:41.451161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:41.451161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:41.451161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:41.451161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:41.451161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:41.451161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:41.451161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:41.452278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:41.452278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:41.452278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:41.452278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:41.452278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:41.452278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:41.452278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:41.452278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:41.452278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:41.452278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:41.452278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:41.452278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:41.452278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:41.452278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:41.452278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:41.452278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:41.453252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:41.453252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:41.453252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:41.453252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:41.453252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:41.453252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:41.453252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:41.453252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:41.453252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:41.453252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:41.453252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 15:34:41.453252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:41.455689 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 231/479] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:34:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:41.461292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35138807-c7fc-4734-b977-c1338232a9c9 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:41.582245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35138807-c7fc-4734-b977-c1338232a9c9 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:41.618798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd76932-4d81-4082-bb39-948b5f1b9d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:41.619760 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd76932-4d81-4082-bb39-948b5f1b9d11 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:41.620193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd76932-4d81-4082-bb39-948b5f1b9d11 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:41.620626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd76932-4d81-4082-bb39-948b5f1b9d11 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:41.621530 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd76932-4d81-4082-bb39-948b5f1b9d11 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:41.622010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd76932-4d81-4082-bb39-948b5f1b9d11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:41.623556 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd76932-4d81-4082-bb39-948b5f1b9d11 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:41.632669 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 249/480] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:42.523213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e604d-29ea-4f38-acc9-54adfb43249b None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:42.523727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e604d-29ea-4f38-acc9-54adfb43249b None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:42.523834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e604d-29ea-4f38-acc9-54adfb43249b None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:42.526246 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 232/481] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:42.531901 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a695b0-6314-4122-a2c6-c518b1020547 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:42.532776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a695b0-6314-4122-a2c6-c518b1020547 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:42.533256 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a695b0-6314-4122-a2c6-c518b1020547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:42.542572 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:42.558755 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-55a695b0-6314-4122-a2c6-c518b1020547 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:42.560325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-55a695b0-6314-4122-a2c6-c518b1020547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:42.647935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 250/482] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:42.691131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e604d-29ea-4f38-acc9-54adfb43249b None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:42.847519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592e604d-29ea-4f38-acc9-54adfb43249b None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:42.869969 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:42.898570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44498739-2166-45be-8b31-7874ad5bd449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:42.899385 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44498739-2166-45be-8b31-7874ad5bd449 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:42.900222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44498739-2166-45be-8b31-7874ad5bd449 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:42.900504 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44498739-2166-45be-8b31-7874ad5bd449 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:42.918603 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44498739-2166-45be-8b31-7874ad5bd449 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:42.926592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-44498739-2166-45be-8b31-7874ad5bd449 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:42.926592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:42.926592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:42.926592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:42.926592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:42.926592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:42.926592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:42.926592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:42.926592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:42.926592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:42.926592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:42.926592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:42.926592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:42.926592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:42.926592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:42.926592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:42.927623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:42.927623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:42.927623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:42.927623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:42.927623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:42.927623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:42.927623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:42.927623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:42.927623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:42.927623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:42.927623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:42.927623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:42.927623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:42.927623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:42.927623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:42.927623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:42.928416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:42.928416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:42.928416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:42.928416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:42.928416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:42.928416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:42.928416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:42.928416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:42.928416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:42.928416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:42.928416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:42.928416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:42.929532 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 233/483] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:34:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:42.934974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a695b0-6314-4122-a2c6-c518b1020547 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:43.073023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a695b0-6314-4122-a2c6-c518b1020547 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:43.113093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763ee07-5473-43b9-8010-a64046ddd81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:43.113791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763ee07-5473-43b9-8010-a64046ddd81d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:43.114070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763ee07-5473-43b9-8010-a64046ddd81d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:43.114344 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763ee07-5473-43b9-8010-a64046ddd81d None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:43.115092 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763ee07-5473-43b9-8010-a64046ddd81d None admin] RBAC: Adding query filter params (name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:43.115397 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763ee07-5473-43b9-8010-a64046ddd81d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:43.116759 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763ee07-5473-43b9-8010-a64046ddd81d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:43.118835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:43.129047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 251/484] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:34:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:43.135632 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44498739-2166-45be-8b31-7874ad5bd449 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:43.286164 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44498739-2166-45be-8b31-7874ad5bd449 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:43.336819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42daec8-2138-4565-a6c9-753c39125a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:43.342388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42daec8-2138-4565-a6c9-753c39125a23 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:43.342742 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42daec8-2138-4565-a6c9-753c39125a23 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:43.343034 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42daec8-2138-4565-a6c9-753c39125a23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:43.355155 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42daec8-2138-4565-a6c9-753c39125a23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:43.356657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42daec8-2138-4565-a6c9-753c39125a23 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:43.378494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-d42daec8-2138-4565-a6c9-753c39125a23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:43.378494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:43.378494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:43.378494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:43.378494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:43.378494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:43.378494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:43.378494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:43.378494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:43.378494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:43.378494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:43.378494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:43.378494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:43.378494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:43.378494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:43.379374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:43.379374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:43.379374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:43.379374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:43.379374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:43.379374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:43.379374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:43.379374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:43.379374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:43.379374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:43.379374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:43.379374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:43.379374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:43.379374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:43.379374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:43.379374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:43.379374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:43.380194 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:43.380194 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:43.380194 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:43.380194 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:43.383501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 234/485] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:43.385901 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a39f5-1833-4a9f-885c-a6a3acf9d17e None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:43.386189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a39f5-1833-4a9f-885c-a6a3acf9d17e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:43.386446 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42daec8-2138-4565-a6c9-753c39125a23 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:43.386566 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a39f5-1833-4a9f-885c-a6a3acf9d17e None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:43.388222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 252/486] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:43.394738 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2c07d-637a-48fe-afe7-7c27fa798b63 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:43.395034 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2c07d-637a-48fe-afe7-7c27fa798b63 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:43.395326 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2c07d-637a-48fe-afe7-7c27fa798b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:43.418619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:43.435024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-8ff2c07d-637a-48fe-afe7-7c27fa798b63 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:43.436235 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8ff2c07d-637a-48fe-afe7-7c27fa798b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:43.522879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 253/487] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:43.542530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42daec8-2138-4565-a6c9-753c39125a23 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:43.551692 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2c07d-637a-48fe-afe7-7c27fa798b63 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:43.612899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb5e04-7c61-4a99-a69c-2c90ce464b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:43.613625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb5e04-7c61-4a99-a69c-2c90ce464b3d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:43.613995 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb5e04-7c61-4a99-a69c-2c90ce464b3d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:43.614278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb5e04-7c61-4a99-a69c-2c90ce464b3d None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:43.615017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb5e04-7c61-4a99-a69c-2c90ce464b3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:43.615333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb5e04-7c61-4a99-a69c-2c90ce464b3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:43.616735 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb5e04-7c61-4a99-a69c-2c90ce464b3d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:43.638585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 235/488] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:43.642716 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb5e04-7c61-4a99-a69c-2c90ce464b3d None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:43.711165 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2c07d-637a-48fe-afe7-7c27fa798b63 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:43.746635 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:43.756639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f2650-5f85-4993-8463-cfe726aa780b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:43.757435 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f2650-5f85-4993-8463-cfe726aa780b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:43.757759 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f2650-5f85-4993-8463-cfe726aa780b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:43.758328 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f2650-5f85-4993-8463-cfe726aa780b None admin] PATH_INFO: `/v3/roles/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:43.759776 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:43.773795 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5f2650-5f85-4993-8463-cfe726aa780b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:43.779189 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb5e04-7c61-4a99-a69c-2c90ce464b3d None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:43.785761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-ff5f2650-5f85-4993-8463-cfe726aa780b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:43.785761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:43.785761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:43.785761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:43.785761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:43.785761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:43.785761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:43.785761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:43.785761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:43.785761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:43.785761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:43.785761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:43.785761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:43.785761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:43.785761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:43.785761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:43.786884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:43.786884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:43.786884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:43.786884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:43.786884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:43.786884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:43.786884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:43.786884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:43.786884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:43.786884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:43.786884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:43.786884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:43.786884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:43.786884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:43.786884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:43.786884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:43.787719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:43.787719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:43.787719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:43.787719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:43.787719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:43.787719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:43.787719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:43.787719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:43.787719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:43.787719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:43.787719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:43.787719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:43.787719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 254/489] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:34:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:34:43.788926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820f7d7-15fd-43f6-a4a3-8b38e2f4e0b8 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:43.789149 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820f7d7-15fd-43f6-a4a3-8b38e2f4e0b8 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:43.789507 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820f7d7-15fd-43f6-a4a3-8b38e2f4e0b8 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:43.796224 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 255/490] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:43.797024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820f7d7-15fd-43f6-a4a3-8b38e2f4e0b8 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:43.822500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5152a-6b7e-4b0e-a637-fe3d729ba6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:43.823676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5152a-6b7e-4b0e-a637-fe3d729ba6d5 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:43.824189 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5152a-6b7e-4b0e-a637-fe3d729ba6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:43.824892 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5152a-6b7e-4b0e-a637-fe3d729ba6d5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:43.832735 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a5152a-6b7e-4b0e-a637-fe3d729ba6d5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:43.834432 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a5152a-6b7e-4b0e-a637-fe3d729ba6d5 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:43.851618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-57a5152a-6b7e-4b0e-a637-fe3d729ba6d5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:34:43.851618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:43.851618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:43.851618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:43.851618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:43.851618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:43.851618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:43.851618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:43.851618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:43.851618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:43.851618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:43.851618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:43.851618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:34:43.851618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:34:43.851618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:34:43.851618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:34:43.852622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:43.852622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:43.852622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:34:43.852622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:43.852622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:34:43.852622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:43.852622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:43.852622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:43.852622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:43.852622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:43.852622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:43.852622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:43.852622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:43.852622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:43.852622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:43.852622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:43.852622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:43.852622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:43.854624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:43.854624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:43.854624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:34:43.854624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:34:43.854624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:34:43.854624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:34:43.854624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:34:43.854624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:34:43.854624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:34:43.854624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:43.857381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 236/491] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Jun 27 15:34:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:43.858545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5be13-3238-4e76-ae4e-0f047ffe69cc None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:43.858844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5be13-3238-4e76-ae4e-0f047ffe69cc None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:43.859124 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5be13-3238-4e76-ae4e-0f047ffe69cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:43.876698 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-03c5be13-3238-4e76-ae4e-0f047ffe69cc None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:43.877950 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-03c5be13-3238-4e76-ae4e-0f047ffe69cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:43.927213 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820f7d7-15fd-43f6-a4a3-8b38e2f4e0b8 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:43.965663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 237/492] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:43.965663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5be13-3238-4e76-ae4e-0f047ffe69cc None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:43.979879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6e7f3-89a6-4f69-b58b-9bd64f449542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:43.980963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6e7f3-89a6-4f69-b58b-9bd64f449542 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:43.981598 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6e7f3-89a6-4f69-b58b-9bd64f449542 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:43.982077 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6e7f3-89a6-4f69-b58b-9bd64f449542 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:43.983128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6e7f3-89a6-4f69-b58b-9bd64f449542 None admin] RBAC: Adding query filter params (name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:43.983648 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6e7f3-89a6-4f69-b58b-9bd64f449542 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:43.985270 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6e7f3-89a6-4f69-b58b-9bd64f449542 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:43.995522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 256/493] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:34:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:44.000803 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6e7f3-89a6-4f69-b58b-9bd64f449542 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:44.055284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:44.076531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:44.105145 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5be13-3238-4e76-ae4e-0f047ffe69cc None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:44.120384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6e7f3-89a6-4f69-b58b-9bd64f449542 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:44.154660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75def67-5f88-4363-b7f7-e46c54311f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:44.155451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75def67-5f88-4363-b7f7-e46c54311f06 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:44.155730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75def67-5f88-4363-b7f7-e46c54311f06 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:44.156005 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75def67-5f88-4363-b7f7-e46c54311f06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:44.167921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a2614-f8a4-4d0c-ad43-261da6d17fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:44.168478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75def67-5f88-4363-b7f7-e46c54311f06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:44.168645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a2614-f8a4-4d0c-ad43-261da6d17fc6 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:44.168899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a2614-f8a4-4d0c-ad43-261da6d17fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:44.169178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a2614-f8a4-4d0c-ad43-261da6d17fc6 None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:44.169951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75def67-5f88-4363-b7f7-e46c54311f06 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:44.170111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a2614-f8a4-4d0c-ad43-261da6d17fc6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:44.170380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a2614-f8a4-4d0c-ad43-261da6d17fc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:44.171826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a2614-f8a4-4d0c-ad43-261da6d17fc6 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:44.188463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-c75def67-5f88-4363-b7f7-e46c54311f06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:44.188463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:44.188463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:44.188463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:44.188463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:44.188463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:44.188463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:44.188463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:44.188463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:44.188463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:44.188463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:44.188463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:44.188463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:44.188463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:44.188463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:44.189321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:44.189321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:44.189321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:44.189321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:44.189321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:44.189321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:44.189321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:44.189321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:44.189321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:44.189321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:44.189321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:44.189321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:44.189321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:44.189321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:44.189321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:44.189321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:44.189321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:44.190205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:44.190205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:44.190205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:44.190205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:44.193912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 257/494] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:34:44.195025 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75def67-5f88-4363-b7f7-e46c54311f06 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:44.214386 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 238/495] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Jun 27 15:34:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:44.215963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3575b66-e1bf-4f33-a8ca-c070d3d93397 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:44.216340 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3575b66-e1bf-4f33-a8ca-c070d3d93397 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:44.216537 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3575b66-e1bf-4f33-a8ca-c070d3d93397 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:44.218472 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 239/496] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:44.219211 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3575b66-e1bf-4f33-a8ca-c070d3d93397 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:44.312066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:44.315264 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75def67-5f88-4363-b7f7-e46c54311f06 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:44.334108 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3575b66-e1bf-4f33-a8ca-c070d3d93397 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:44.355040 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3866ee2-d137-48eb-af33-2d90e3f9d9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:44.355844 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3866ee2-d137-48eb-af33-2d90e3f9d9ff None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:44.356129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3866ee2-d137-48eb-af33-2d90e3f9d9ff None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:44.356841 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3866ee2-d137-48eb-af33-2d90e3f9d9ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:44.367954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3866ee2-d137-48eb-af33-2d90e3f9d9ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:44.370424 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8698d-8b46-4359-8510-dd47a7283cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:44.371093 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8698d-8b46-4359-8510-dd47a7283cd0 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:44.371380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8698d-8b46-4359-8510-dd47a7283cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:44.371659 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8698d-8b46-4359-8510-dd47a7283cd0 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:44.372388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8698d-8b46-4359-8510-dd47a7283cd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:44.372702 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8698d-8b46-4359-8510-dd47a7283cd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:44.374150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8698d-8b46-4359-8510-dd47a7283cd0 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:44.375704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-f3866ee2-d137-48eb-af33-2d90e3f9d9ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:44.375704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:44.375704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:44.375704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:44.375704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:44.375704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:44.375704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:44.375704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:44.375704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:44.375704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:44.375704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:44.375704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:44.375704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:44.375704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:44.375704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:44.375704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:44.376573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:44.376573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:44.376573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:44.376573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:44.376573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:44.376573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:44.376573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:44.376573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:44.376573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:44.376573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:44.376573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:44.376573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:44.376573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:44.376573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:44.376573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:44.376573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:44.377611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:44.377611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:44.377611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:44.377611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:44.377611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:44.377611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:44.377611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:44.377611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:44.377611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:44.377611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:44.377611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:44.377611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:44.378321 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:44.378390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 258/497] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:34:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:44.378792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3866ee2-d137-48eb-af33-2d90e3f9d9ff None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:44.391264 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 240/498] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:44.392806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53690717-1d9d-4d87-a6c2-2e7099d091c7 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:44.393735 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53690717-1d9d-4d87-a6c2-2e7099d091c7 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:44.394187 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53690717-1d9d-4d87-a6c2-2e7099d091c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:44.416791 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-53690717-1d9d-4d87-a6c2-2e7099d091c7 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:44.418614 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-53690717-1d9d-4d87-a6c2-2e7099d091c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:44.488217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3866ee2-d137-48eb-af33-2d90e3f9d9ff None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:44.489977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 241/499] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:44.490810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53690717-1d9d-4d87-a6c2-2e7099d091c7 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:44.535417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ca72c-baff-41b0-9e33-90006db3435b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:44.536066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ca72c-baff-41b0-9e33-90006db3435b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:44.536344 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ca72c-baff-41b0-9e33-90006db3435b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:44.536616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ca72c-baff-41b0-9e33-90006db3435b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:44.565028 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:44.592408 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53690717-1d9d-4d87-a6c2-2e7099d091c7 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:44.605192 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ca72c-baff-41b0-9e33-90006db3435b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:44.606677 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ca72c-baff-41b0-9e33-90006db3435b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:44.616143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:44.626708 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518abd92-9673-42de-ab49-dbb79566c6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:44.627378 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518abd92-9673-42de-ab49-dbb79566c6f5 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:44.627669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518abd92-9673-42de-ab49-dbb79566c6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:44.627945 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518abd92-9673-42de-ab49-dbb79566c6f5 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:44.628688 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518abd92-9673-42de-ab49-dbb79566c6f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:44.628994 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518abd92-9673-42de-ab49-dbb79566c6f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:44.630418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518abd92-9673-42de-ab49-dbb79566c6f5 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:44.638671 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 242/500] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:34:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:44.639466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518abd92-9673-42de-ab49-dbb79566c6f5 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:44.707269 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 259/501] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:44.708585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf4db9-984d-4376-a823-456db48a94cb None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:44.708871 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf4db9-984d-4376-a823-456db48a94cb None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:44.709148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf4db9-984d-4376-a823-456db48a94cb None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:44.712815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 260/502] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:44.712815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf4db9-984d-4376-a823-456db48a94cb None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:44.739516 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518abd92-9673-42de-ab49-dbb79566c6f5 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:44.771308 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:44.774383 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f1a44-33fc-4d05-b56f-9c29b7343514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:44.775361 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f1a44-33fc-4d05-b56f-9c29b7343514 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:44.775588 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f1a44-33fc-4d05-b56f-9c29b7343514 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:44.775871 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f1a44-33fc-4d05-b56f-9c29b7343514 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:44.784770 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f1a44-33fc-4d05-b56f-9c29b7343514 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:44.786286 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f1a44-33fc-4d05-b56f-9c29b7343514 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:44.801419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-669f1a44-33fc-4d05-b56f-9c29b7343514 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 15:34:44.801419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:44.801419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:44.801419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:44.801419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:44.801419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:44.801419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:44.801419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:44.801419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:44.801419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:44.801419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:44.801419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:44.801419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:34:44.801419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:34:44.801419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:34:44.801419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:34:44.802368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:44.802368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:44.802368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:34:44.802368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:44.802368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:34:44.802368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:44.802368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:44.802368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:44.802368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:44.802368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:44.802368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:44.802368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:44.802368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:44.802368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:44.802368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:44.802368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:44.802368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:44.802368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:44.803320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:44.803320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:44.803320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:34:44.803320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:34:44.803320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:34:44.803320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:34:44.803320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:34:44.803320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:34:44.803320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 15:34:44.803320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:44.807878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 243/503] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:44.808617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f1a44-33fc-4d05-b56f-9c29b7343514 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:44.818138 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf4db9-984d-4376-a823-456db48a94cb None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:44.855161 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbf05c-5c60-41c5-a78d-1945cc839553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:44.855958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbf05c-5c60-41c5-a78d-1945cc839553 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:44.856237 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbf05c-5c60-41c5-a78d-1945cc839553 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:44.856516 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbf05c-5c60-41c5-a78d-1945cc839553 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:44.856669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:44.867298 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fbf05c-5c60-41c5-a78d-1945cc839553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:44.875593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-91fbf05c-5c60-41c5-a78d-1945cc839553 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:44.875593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:44.875593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:44.875593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:44.875593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:44.875593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:44.875593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:44.875593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:44.875593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:44.875593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:44.875593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:44.875593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:44.875593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:44.875593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:44.875593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:44.875593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:44.876532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:44.876532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:44.876532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:44.876532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:44.876532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:44.876532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:44.876532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:44.876532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:44.876532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:44.876532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:44.876532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:44.876532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:44.876532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:44.876532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:44.876532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:44.876532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:44.877335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:44.877335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:44.877335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:44.877335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:44.877335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:44.877335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:44.877335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:44.877335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:44.877335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:44.877335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:44.877335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:44.877335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:44.877335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 261/504] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:34:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:44.878032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbf05c-5c60-41c5-a78d-1945cc839553 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:44.916267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f1a44-33fc-4d05-b56f-9c29b7343514 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:44.949916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2537cd-b173-40e9-b902-9ec63c5027dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:44.950722 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2537cd-b173-40e9-b902-9ec63c5027dd None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:44.951009 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2537cd-b173-40e9-b902-9ec63c5027dd None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:44.951284 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2537cd-b173-40e9-b902-9ec63c5027dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:44.961877 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2537cd-b173-40e9-b902-9ec63c5027dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:44.963491 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2537cd-b173-40e9-b902-9ec63c5027dd None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:44.980936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-5c2537cd-b173-40e9-b902-9ec63c5027dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:44.980936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:44.980936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:44.980936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:44.980936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:44.980936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:44.980936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:44.980936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:44.980936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:44.980936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:44.980936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:44.980936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:44.980936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:44.980936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:44.980936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:44.981923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:44.981923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:44.981923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:44.981923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:44.981923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:44.981923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:44.981923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:44.981923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:44.981923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:44.981923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:44.981923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:44.981923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:44.981923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:44.981923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:44.981923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:44.981923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:44.981923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:44.982749 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:44.982749 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:44.982749 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:44.982749 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:44.982749 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 244/505] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:44.983024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbf05c-5c60-41c5-a78d-1945cc839553 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:44.983367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53921ecc-960f-4b67-bd78-f7c67b19bb53 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:44.983607 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53921ecc-960f-4b67-bd78-f7c67b19bb53 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:44.983882 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53921ecc-960f-4b67-bd78-f7c67b19bb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:45.002326 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-53921ecc-960f-4b67-bd78-f7c67b19bb53 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:45.003242 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:45.003442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-53921ecc-960f-4b67-bd78-f7c67b19bb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:45.017873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d285b7-1866-4aff-a641-740e94810638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:45.018673 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d285b7-1866-4aff-a641-740e94810638 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:45.019046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d285b7-1866-4aff-a641-740e94810638 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:45.019367 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d285b7-1866-4aff-a641-740e94810638 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:45.030274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d285b7-1866-4aff-a641-740e94810638 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:45.031743 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d285b7-1866-4aff-a641-740e94810638 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:45.049670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-64d285b7-1866-4aff-a641-740e94810638 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:45.049670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:45.049670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:45.049670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:45.049670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:45.049670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:45.049670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:45.049670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:45.049670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:45.049670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:45.049670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:45.049670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:45.049670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:45.049670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:45.049670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:45.050597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:45.050597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:45.050597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:45.050597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:45.050597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:45.050597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:45.050597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:45.050597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:45.050597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:45.050597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:45.050597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:45.050597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:45.050597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:45.050597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:45.050597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:45.050597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:45.050597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:45.051489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:45.051489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:45.051489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:45.051489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:45.051489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 262/506] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:45.051743 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d285b7-1866-4aff-a641-740e94810638 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:45.075615 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 245/507] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:45.076457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53921ecc-960f-4b67-bd78-f7c67b19bb53 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:45.116762 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:45.159640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d285b7-1866-4aff-a641-740e94810638 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:45.192582 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53921ecc-960f-4b67-bd78-f7c67b19bb53 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:45.193486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e3b82-7738-4756-a443-907c7b6f146b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:45.194196 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e3b82-7738-4756-a443-907c7b6f146b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:45.194479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e3b82-7738-4756-a443-907c7b6f146b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:45.194770 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e3b82-7738-4756-a443-907c7b6f146b None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:45.195597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e3b82-7738-4756-a443-907c7b6f146b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:45.195906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e3b82-7738-4756-a443-907c7b6f146b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:45.198212 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e3b82-7738-4756-a443-907c7b6f146b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:45.227776 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda477f6-6418-45cc-838a-b117e874a5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:45.228451 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda477f6-6418-45cc-838a-b117e874a5d0 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:45.228739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda477f6-6418-45cc-838a-b117e874a5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:45.229018 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda477f6-6418-45cc-838a-b117e874a5d0 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:45.229815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda477f6-6418-45cc-838a-b117e874a5d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:45.230129 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda477f6-6418-45cc-838a-b117e874a5d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:45.231528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda477f6-6418-45cc-838a-b117e874a5d0 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:45.235087 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 263/508] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Jun 27 15:34:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:45.235930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e3b82-7738-4756-a443-907c7b6f146b None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:45.245506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 246/509] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:34:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:45.245506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda477f6-6418-45cc-838a-b117e874a5d0 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:45.268385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:45.335621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:45.338088 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e3b82-7738-4756-a443-907c7b6f146b None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:45.345247 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda477f6-6418-45cc-838a-b117e874a5d0 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:45.372633 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681ab74-6c9a-4c64-b5ff-b67235111f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:45.373472 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681ab74-6c9a-4c64-b5ff-b67235111f9a None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:45.373791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681ab74-6c9a-4c64-b5ff-b67235111f9a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:45.374074 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681ab74-6c9a-4c64-b5ff-b67235111f9a None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:45.374802 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681ab74-6c9a-4c64-b5ff-b67235111f9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:45.375110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681ab74-6c9a-4c64-b5ff-b67235111f9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:45.376483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681ab74-6c9a-4c64-b5ff-b67235111f9a None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:45.380171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad68e8-a51a-4213-bbe1-2c04d501dc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:45.380840 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad68e8-a51a-4213-bbe1-2c04d501dc7e None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:45.381127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad68e8-a51a-4213-bbe1-2c04d501dc7e None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:45.381416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad68e8-a51a-4213-bbe1-2c04d501dc7e None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:45.382167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad68e8-a51a-4213-bbe1-2c04d501dc7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:45.382477 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad68e8-a51a-4213-bbe1-2c04d501dc7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:45.385588 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad68e8-a51a-4213-bbe1-2c04d501dc7e None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:45.388287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 264/510] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:45.389080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681ab74-6c9a-4c64-b5ff-b67235111f9a None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:45.403926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 247/511] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:45.403926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad68e8-a51a-4213-bbe1-2c04d501dc7e None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:45.483218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:45.493395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681ab74-6c9a-4c64-b5ff-b67235111f9a None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:45.502746 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad68e8-a51a-4213-bbe1-2c04d501dc7e None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:45.528949 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5282f4-8183-41ae-9656-b4cb1cc018b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:45.529721 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5282f4-8183-41ae-9656-b4cb1cc018b8 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:45.529976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5282f4-8183-41ae-9656-b4cb1cc018b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:45.530258 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5282f4-8183-41ae-9656-b4cb1cc018b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:45.542018 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008deac1-c40d-4901-a3e2-9e6c9a6b8bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:45.542237 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5282f4-8183-41ae-9656-b4cb1cc018b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:45.542691 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008deac1-c40d-4901-a3e2-9e6c9a6b8bf1 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:45.542980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008deac1-c40d-4901-a3e2-9e6c9a6b8bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:45.543260 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008deac1-c40d-4901-a3e2-9e6c9a6b8bf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:45.543652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5282f4-8183-41ae-9656-b4cb1cc018b8 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:45.563242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:45.563544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-ac5282f4-8183-41ae-9656-b4cb1cc018b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:45.563544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:45.563544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:45.563544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:45.563544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:45.563544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:45.563544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:45.563544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:45.563544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:45.563544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:45.563544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:45.563544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:45.563544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:45.563544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:45.563544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:45.564388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:45.564388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:45.564388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:45.564388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:45.564388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:45.564388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:45.564388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:45.564388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:45.564388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:45.564388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:45.564388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:45.564388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:45.564388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:45.564388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:45.564388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:45.564388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:45.564388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:45.565906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:45.565906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:45.565906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:45.565906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:45.565906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 265/512] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:45.566211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5282f4-8183-41ae-9656-b4cb1cc018b8 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:45.615285 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008deac1-c40d-4901-a3e2-9e6c9a6b8bf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:45.616771 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008deac1-c40d-4901-a3e2-9e6c9a6b8bf1 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:45.676609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5282f4-8183-41ae-9656-b4cb1cc018b8 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:45.710860 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0e619-e903-4f99-94b7-0546c7ea006c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:45.711653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0e619-e903-4f99-94b7-0546c7ea006c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:45.712062 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0e619-e903-4f99-94b7-0546c7ea006c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:45.713368 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0e619-e903-4f99-94b7-0546c7ea006c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:45.717414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 248/513] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:45.718281 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008deac1-c40d-4901-a3e2-9e6c9a6b8bf1 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:45.726257 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0e619-e903-4f99-94b7-0546c7ea006c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:45.727739 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0e619-e903-4f99-94b7-0546c7ea006c None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:45.730177 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:45.747852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-0dd0e619-e903-4f99-94b7-0546c7ea006c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:45.747852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:45.747852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:45.747852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:45.747852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:45.747852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:45.747852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:45.747852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:45.747852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:45.747852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:45.747852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:45.747852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:45.747852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:45.747852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:45.747852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:45.748741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:45.748741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:45.748741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:45.748741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:45.748741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:45.748741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:45.748741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:45.748741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:45.748741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:45.748741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:45.748741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:45.748741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:45.748741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:45.748741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:45.748741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:45.748741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:45.748741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:45.749545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:45.749545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:45.749545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:45.749545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:45.753779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 266/514] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:45.754391 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0e619-e903-4f99-94b7-0546c7ea006c None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:45.771036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:45.824154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008deac1-c40d-4901-a3e2-9e6c9a6b8bf1 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:45.858424 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0e619-e903-4f99-94b7-0546c7ea006c None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:45.860333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ba9e5-b8bc-4533-975d-7960f379354d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:45.861165 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ba9e5-b8bc-4533-975d-7960f379354d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:45.861448 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ba9e5-b8bc-4533-975d-7960f379354d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:45.861756 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ba9e5-b8bc-4533-975d-7960f379354d None admin] PATH_INFO: `/v3/users/glance` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:45.870606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ba9e5-b8bc-4533-975d-7960f379354d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:45.872099 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ba9e5-b8bc-4533-975d-7960f379354d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:45.888298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-f34ba9e5-b8bc-4533-975d-7960f379354d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 27 15:34:45.888298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:45.888298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:45.888298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:45.888298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:45.888298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:45.888298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:45.888298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:45.888298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:45.888298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:45.888298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:45.888298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:45.888298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:34:45.888298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:34:45.888298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:34:45.888298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:34:45.889339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:45.889339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:45.889339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:34:45.889339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:45.889339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:34:45.889339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:45.889339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:45.889339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:45.889339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:45.889339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:45.889339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:45.889339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:45.889339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:45.889339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:45.889339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:45.889339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:45.889339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:45.889339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:45.890241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:45.890241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:45.890241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:34:45.890241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:34:45.890241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:34:45.890241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:34:45.890241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:34:45.890241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:34:45.890241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 27 15:34:45.890241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:45.890241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 249/515] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:45.890796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ba9e5-b8bc-4533-975d-7960f379354d None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:45.900709 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d911a2b-e701-4b16-b463-3ca0b35aec66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:45.901534 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d911a2b-e701-4b16-b463-3ca0b35aec66 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:45.901873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d911a2b-e701-4b16-b463-3ca0b35aec66 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:45.902154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d911a2b-e701-4b16-b463-3ca0b35aec66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:45.909836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d911a2b-e701-4b16-b463-3ca0b35aec66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:45.911334 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d911a2b-e701-4b16-b463-3ca0b35aec66 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:45.928217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-0d911a2b-e701-4b16-b463-3ca0b35aec66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:45.928217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:45.928217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:45.928217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:45.928217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:45.928217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:45.928217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:45.928217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:45.928217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:45.928217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:45.928217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:45.928217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:45.928217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:34:45.928217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:34:45.928217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:34:45.932004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:34:45.932004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:45.932004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:45.932004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:45.932004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:45.932004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:45.932004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:45.932004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:45.932004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:45.932004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:45.932004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:45.932004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:45.932004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:45.932004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:45.932004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:45.932004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:45.932004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:45.934149 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:34:45.934149 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:34:45.934149 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:34:45.934149 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:34:45.934149 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:34:45.934149 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:34:45.934149 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:45.934149 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:45.934149 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 267/516] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:45.934149 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d911a2b-e701-4b16-b463-3ca0b35aec66 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:45.996206 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:46.012223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ba9e5-b8bc-4533-975d-7960f379354d None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:46.024299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:46.046601 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d911a2b-e701-4b16-b463-3ca0b35aec66 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:46.051189 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee03487-ffce-4914-bcd2-4f5b086d67e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:46.051866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee03487-ffce-4914-bcd2-4f5b086d67e4 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:46.052151 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee03487-ffce-4914-bcd2-4f5b086d67e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:46.052435 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee03487-ffce-4914-bcd2-4f5b086d67e4 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:46.053162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee03487-ffce-4914-bcd2-4f5b086d67e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:46.053473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee03487-ffce-4914-bcd2-4f5b086d67e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:46.054911 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee03487-ffce-4914-bcd2-4f5b086d67e4 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:46.067782 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 250/517] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:46.068625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee03487-ffce-4914-bcd2-4f5b086d67e4 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:46.084381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587f446-2fb1-4446-ad2a-1c6696e2f0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:46.085184 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587f446-2fb1-4446-ad2a-1c6696e2f0ad None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:46.085466 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587f446-2fb1-4446-ad2a-1c6696e2f0ad None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:46.085771 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587f446-2fb1-4446-ad2a-1c6696e2f0ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:46.097310 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587f446-2fb1-4446-ad2a-1c6696e2f0ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:46.098996 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587f446-2fb1-4446-ad2a-1c6696e2f0ad None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:46.117160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-a587f446-2fb1-4446-ad2a-1c6696e2f0ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:46.117160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:46.117160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:46.117160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:46.117160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:46.117160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:46.117160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:46.117160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:46.117160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:46.117160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:46.117160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:46.117160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:46.117160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:46.117160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:46.117160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:46.118174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:46.118174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:46.118174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:46.118174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:46.118174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:46.118174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:46.118174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:46.118174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:46.118174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:46.118174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:46.118174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:46.118174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:46.118174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:46.118174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:46.118174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:46.118174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:46.118174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:46.119800 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:46.119800 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:46.119800 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:46.119800 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:46.119800 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 268/518] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:46.120118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587f446-2fb1-4446-ad2a-1c6696e2f0ad None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:46.180572 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee03487-ffce-4914-bcd2-4f5b086d67e4 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:46.217470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0454d5-f7c1-4271-9862-220e9aad5753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:46.218170 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0454d5-f7c1-4271-9862-220e9aad5753 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:46.218455 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0454d5-f7c1-4271-9862-220e9aad5753 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:46.218737 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0454d5-f7c1-4271-9862-220e9aad5753 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:46.219469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0454d5-f7c1-4271-9862-220e9aad5753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:46.219777 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0454d5-f7c1-4271-9862-220e9aad5753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:46.221176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0454d5-f7c1-4271-9862-220e9aad5753 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:46.232167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587f446-2fb1-4446-ad2a-1c6696e2f0ad None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:46.237897 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 251/519] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:46.239292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0454d5-f7c1-4271-9862-220e9aad5753 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:46.246494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:46.273728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:46.276788 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c263b03-209e-4db1-9a81-f2c12a36113c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:46.277470 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c263b03-209e-4db1-9a81-f2c12a36113c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:46.277795 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c263b03-209e-4db1-9a81-f2c12a36113c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:46.278072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c263b03-209e-4db1-9a81-f2c12a36113c None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:46.278919 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c263b03-209e-4db1-9a81-f2c12a36113c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:46.279230 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c263b03-209e-4db1-9a81-f2c12a36113c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:46.280680 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c263b03-209e-4db1-9a81-f2c12a36113c None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:46.299673 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 269/520] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Jun 27 15:34:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:46.300490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c263b03-209e-4db1-9a81-f2c12a36113c None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:46.353994 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0454d5-f7c1-4271-9862-220e9aad5753 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:46.388954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae13e3-a00e-42da-a96f-6ce677e41f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:46.389630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae13e3-a00e-42da-a96f-6ce677e41f02 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:46.389941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae13e3-a00e-42da-a96f-6ce677e41f02 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:46.390220 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae13e3-a00e-42da-a96f-6ce677e41f02 None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:46.391016 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae13e3-a00e-42da-a96f-6ce677e41f02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:46.391331 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae13e3-a00e-42da-a96f-6ce677e41f02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:46.392774 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae13e3-a00e-42da-a96f-6ce677e41f02 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:46.404116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c263b03-209e-4db1-9a81-f2c12a36113c None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:46.411072 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 252/521] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:34:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:46.411905 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae13e3-a00e-42da-a96f-6ce677e41f02 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:46.438341 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ad129-7bb4-4555-b761-aabfa8f6fac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:46.439009 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ad129-7bb4-4555-b761-aabfa8f6fac7 None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:46.439294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ad129-7bb4-4555-b761-aabfa8f6fac7 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:46.439568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ad129-7bb4-4555-b761-aabfa8f6fac7 None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:46.440271 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ad129-7bb4-4555-b761-aabfa8f6fac7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:46.441734 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ad129-7bb4-4555-b761-aabfa8f6fac7 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:46.461573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 270/522] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jun 27 15:34:46 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:34:46.462593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ad129-7bb4-4555-b761-aabfa8f6fac7 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:46.488274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:46.508834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:46.525343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae13e3-a00e-42da-a96f-6ce677e41f02 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:46.570863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeafbac-94ed-4c7f-a977-8e3d8e339138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:46.572056 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeafbac-94ed-4c7f-a977-8e3d8e339138 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:46.572392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeafbac-94ed-4c7f-a977-8e3d8e339138 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:46.572670 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeafbac-94ed-4c7f-a977-8e3d8e339138 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:46.573411 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeafbac-94ed-4c7f-a977-8e3d8e339138 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:46.573771 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeafbac-94ed-4c7f-a977-8e3d8e339138 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:46.575185 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeafbac-94ed-4c7f-a977-8e3d8e339138 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:46.581087 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ad129-7bb4-4555-b761-aabfa8f6fac7 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:46.594970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 253/523] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:46.594970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeafbac-94ed-4c7f-a977-8e3d8e339138 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:46.619475 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb7dcd4-50cc-4ef8-bfcb-042eb72437fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:46.620509 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7dcd4-50cc-4ef8-bfcb-042eb72437fc None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:46.620621 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7dcd4-50cc-4ef8-bfcb-042eb72437fc None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:46.620893 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7dcd4-50cc-4ef8-bfcb-042eb72437fc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:46.628993 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb7dcd4-50cc-4ef8-bfcb-042eb72437fc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:46.630468 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb7dcd4-50cc-4ef8-bfcb-042eb72437fc None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:46.647093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-feb7dcd4-50cc-4ef8-bfcb-042eb72437fc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 15:34:46.647093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:46.647093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:46.647093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:46.647093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:46.647093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:46.647093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:46.647093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:46.647093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:46.647093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:46.647093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:46.647093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:46.647093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:34:46.647093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:34:46.647093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:34:46.647093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:34:46.648047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:46.648047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:46.648047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:34:46.648047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:46.648047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:34:46.648047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:46.648047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:46.648047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:46.648047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:46.648047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:46.648047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:46.648047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:46.648047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:46.648047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:46.648047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:46.648047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:46.648047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:46.648047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:46.648930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:46.648930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:46.648930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:34:46.648930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:34:46.648930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:34:46.648930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:34:46.648930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:34:46.648930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:34:46.648930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 15:34:46.648930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:46.648930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 271/524] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:34:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:46.650353 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb7dcd4-50cc-4ef8-bfcb-042eb72437fc None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:46.717291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeafbac-94ed-4c7f-a977-8e3d8e339138 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:46.747206 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:46.754758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d85b8e-10e9-4935-957f-3b6104e10fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:46.756211 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d85b8e-10e9-4935-957f-3b6104e10fd5 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:46.757008 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d85b8e-10e9-4935-957f-3b6104e10fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:46.757492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d85b8e-10e9-4935-957f-3b6104e10fd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:46.767230 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb7dcd4-50cc-4ef8-bfcb-042eb72437fc None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:46.782919 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:46.816200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d1ad6e-2af9-438f-adfc-f5cd1219fb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:46.817026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1ad6e-2af9-438f-adfc-f5cd1219fb11 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:46.817322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1ad6e-2af9-438f-adfc-f5cd1219fb11 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:46.817910 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1ad6e-2af9-438f-adfc-f5cd1219fb11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:46.818757 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d1ad6e-2af9-438f-adfc-f5cd1219fb11 None admin] RBAC: Adding query filter params (role.id=805851682adf4debbe99b1681c9fcdfc, scope.project.id=7be83fb6f20341139147373837320148, user.id=1c8f6262cf24419daf803ae71aed24f5) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:46.819077 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d1ad6e-2af9-438f-adfc-f5cd1219fb11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:46.821725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d1ad6e-2af9-438f-adfc-f5cd1219fb11 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:46.829444 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 272/525] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:34:46 2022] GET /identity/v3/role_assignments?user.id=1c8f6262cf24419daf803ae71aed24f5&scope.project.id=7be83fb6f20341139147373837320148&role.id=805851682adf4debbe99b1681c9fcdfc => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:46.830176 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d1ad6e-2af9-438f-adfc-f5cd1219fb11 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:46.843374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d85b8e-10e9-4935-957f-3b6104e10fd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:46.845339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d85b8e-10e9-4935-957f-3b6104e10fd5 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:46.951266 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d1ad6e-2af9-438f-adfc-f5cd1219fb11 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:46.965361 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 254/526] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:46.966383 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d85b8e-10e9-4935-957f-3b6104e10fd5 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:46.991032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0daf060-6e59-4934-9b7c-0eb80b84c871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:46.992074 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0daf060-6e59-4934-9b7c-0eb80b84c871 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:46.992502 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0daf060-6e59-4934-9b7c-0eb80b84c871 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:46.992924 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0daf060-6e59-4934-9b7c-0eb80b84c871 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:47.002014 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0daf060-6e59-4934-9b7c-0eb80b84c871 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:47.004037 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0daf060-6e59-4934-9b7c-0eb80b84c871 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:47.021719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-d0daf060-6e59-4934-9b7c-0eb80b84c871 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:47.021719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:47.021719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:47.021719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:47.021719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:47.021719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:47.021719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:47.021719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:47.021719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:47.021719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:47.021719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:47.021719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:47.021719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:34:47.021719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:34:47.021719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:34:47.022642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:34:47.022642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:47.022642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:47.022642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:47.022642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:47.022642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:47.022642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:47.022642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:47.022642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:47.022642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:47.022642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:47.022642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:47.022642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:47.022642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:47.022642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:47.022642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:47.022642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:47.023480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:34:47.023480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:34:47.023480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:34:47.023480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:34:47.023480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:34:47.023480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:34:47.023480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:47.023480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:47.023480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 273/527] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:47.027296 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0daf060-6e59-4934-9b7c-0eb80b84c871 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:47.038725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:47.044237 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:47.130079 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d85b8e-10e9-4935-957f-3b6104e10fd5 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:47.156174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0daf060-6e59-4934-9b7c-0eb80b84c871 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:47.189070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14f6e9-185e-4505-8dd2-e77315aebb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:47.190183 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14f6e9-185e-4505-8dd2-e77315aebb01 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:47.190472 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14f6e9-185e-4505-8dd2-e77315aebb01 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:47.190750 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14f6e9-185e-4505-8dd2-e77315aebb01 None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:47.191879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14f6e9-185e-4505-8dd2-e77315aebb01 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:47.192193 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14f6e9-185e-4505-8dd2-e77315aebb01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:47.194330 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14f6e9-185e-4505-8dd2-e77315aebb01 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:47.199600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0f48c-d81a-4911-9e2e-36f67c9c89f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:47.200708 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0f48c-d81a-4911-9e2e-36f67c9c89f7 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:47.201188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0f48c-d81a-4911-9e2e-36f67c9c89f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:47.201771 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0f48c-d81a-4911-9e2e-36f67c9c89f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:47.214686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0f48c-d81a-4911-9e2e-36f67c9c89f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:47.216496 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0f48c-d81a-4911-9e2e-36f67c9c89f7 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:47.221862 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 255/528] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Jun 27 15:34:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:47.223305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14f6e9-185e-4505-8dd2-e77315aebb01 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:47.240525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-50c0f48c-d81a-4911-9e2e-36f67c9c89f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:47.240525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:47.240525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:47.240525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:47.240525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:47.240525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:47.240525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:47.240525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:47.240525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:47.240525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:47.240525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:47.240525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:47.240525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:47.240525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:47.240525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:47.241437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:47.241437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:47.241437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:47.241437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:47.241437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:47.241437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:47.241437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:47.241437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:47.241437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:47.241437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:47.241437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:47.241437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:47.241437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:47.241437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:47.241437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:47.241437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:47.241437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:47.242275 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:47.242275 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:47.242275 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:47.242275 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:47.250110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 274/529] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:47.251515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0f48c-d81a-4911-9e2e-36f67c9c89f7 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:47.316006 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:47.351414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14f6e9-185e-4505-8dd2-e77315aebb01 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:47.365045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:47.366458 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0f48c-d81a-4911-9e2e-36f67c9c89f7 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:47.404210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fa9ad-7ec1-47c5-a76d-92400ddb2bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:47.405023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fa9ad-7ec1-47c5-a76d-92400ddb2bd3 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:47.405421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fa9ad-7ec1-47c5-a76d-92400ddb2bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:47.405827 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fa9ad-7ec1-47c5-a76d-92400ddb2bd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:47.407225 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63cf5ff-537e-44b4-a2aa-9a4493f2c4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:47.408046 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cf5ff-537e-44b4-a2aa-9a4493f2c4b4 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:47.408471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cf5ff-537e-44b4-a2aa-9a4493f2c4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:47.409111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cf5ff-537e-44b4-a2aa-9a4493f2c4b4 None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:47.411099 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63cf5ff-537e-44b4-a2aa-9a4493f2c4b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:47.411575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63cf5ff-537e-44b4-a2aa-9a4493f2c4b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:47.413193 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63cf5ff-537e-44b4-a2aa-9a4493f2c4b4 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:47.429563 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 256/530] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:34:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:47.431588 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63cf5ff-537e-44b4-a2aa-9a4493f2c4b4 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:47.494261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fa9ad-7ec1-47c5-a76d-92400ddb2bd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:47.495956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fa9ad-7ec1-47c5-a76d-92400ddb2bd3 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:47.560000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:47.570548 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63cf5ff-537e-44b4-a2aa-9a4493f2c4b4 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:47.612290 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684a0dc-ed60-4447-9468-2a5a8710487a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:47.612972 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684a0dc-ed60-4447-9468-2a5a8710487a None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:47.613259 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684a0dc-ed60-4447-9468-2a5a8710487a None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:47.613540 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684a0dc-ed60-4447-9468-2a5a8710487a None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:47.614445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684a0dc-ed60-4447-9468-2a5a8710487a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:47.614763 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684a0dc-ed60-4447-9468-2a5a8710487a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:47.616314 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684a0dc-ed60-4447-9468-2a5a8710487a None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:47.620308 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 275/531] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:47.621846 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fa9ad-7ec1-47c5-a76d-92400ddb2bd3 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:47.630541 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 257/532] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:47.631403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684a0dc-ed60-4447-9468-2a5a8710487a None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:47.669979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:47.746588 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fa9ad-7ec1-47c5-a76d-92400ddb2bd3 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:47.763165 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684a0dc-ed60-4447-9468-2a5a8710487a None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:47.785330 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd9194-0d54-40b8-a3ff-c465c010d298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:47.786107 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd9194-0d54-40b8-a3ff-c465c010d298 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:47.786464 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd9194-0d54-40b8-a3ff-c465c010d298 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:47.786815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd9194-0d54-40b8-a3ff-c465c010d298 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:47.787725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd9194-0d54-40b8-a3ff-c465c010d298 None admin] RBAC: Adding query filter params (role.id=805851682adf4debbe99b1681c9fcdfc, scope.project.id=7be83fb6f20341139147373837320148, user.id=51404afd35b94c47ba5c44f0deb73e30) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:47.788112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd9194-0d54-40b8-a3ff-c465c010d298 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:47.790165 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd9194-0d54-40b8-a3ff-c465c010d298 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:47.806526 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 276/533] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:34:47 2022] GET /identity/v3/role_assignments?user.id=51404afd35b94c47ba5c44f0deb73e30&scope.project.id=7be83fb6f20341139147373837320148&role.id=805851682adf4debbe99b1681c9fcdfc => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:47.807639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd9194-0d54-40b8-a3ff-c465c010d298 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:47.814530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a1106-707b-4752-b129-260691faa0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:47.815631 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a1106-707b-4752-b129-260691faa0a3 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:47.816061 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a1106-707b-4752-b129-260691faa0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:47.816525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a1106-707b-4752-b129-260691faa0a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:47.849973 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a1106-707b-4752-b129-260691faa0a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:47.853510 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a1106-707b-4752-b129-260691faa0a3 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:47.862061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:47.885002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-e32a1106-707b-4752-b129-260691faa0a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:47.885002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:47.885002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:47.885002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:47.885002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:47.885002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:47.885002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:47.885002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:47.885002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:47.885002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:47.885002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:47.885002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:47.885002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:47.885002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:47.885002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:47.885963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:47.885963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:47.885963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:47.885963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:47.885963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:47.885963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:47.885963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:47.885963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:47.885963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:47.885963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:47.885963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:47.885963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:47.885963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:47.885963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:47.885963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:47.885963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:47.885963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:47.886804 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:47.886804 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:47.886804 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:47.886804 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:47.894210 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 258/534] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:47.897270 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a1106-707b-4752-b129-260691faa0a3 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:48.039661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:48.062107 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd9194-0d54-40b8-a3ff-c465c010d298 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:48.083966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a1106-707b-4752-b129-260691faa0a3 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:48.124569 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ded5a8-4afd-40f3-b4f7-e09fba7f0174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:48.125752 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ded5a8-4afd-40f3-b4f7-e09fba7f0174 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:48.126258 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ded5a8-4afd-40f3-b4f7-e09fba7f0174 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:48.126750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ded5a8-4afd-40f3-b4f7-e09fba7f0174 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:48.135109 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ded5a8-4afd-40f3-b4f7-e09fba7f0174 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:48.136946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ded5a8-4afd-40f3-b4f7-e09fba7f0174 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:48.160301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-23ded5a8-4afd-40f3-b4f7-e09fba7f0174 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:48.160301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:48.160301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:48.160301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:48.160301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:48.160301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:48.160301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:48.160301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:48.160301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:48.160301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:48.160301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:48.160301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:48.160301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:34:48.160301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:34:48.160301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:34:48.161207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:34:48.161207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:48.161207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:48.161207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:48.161207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:48.161207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:48.161207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:48.161207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:48.161207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:48.161207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:48.161207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:48.161207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:48.161207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:48.161207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:48.161207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:48.161207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:48.161207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:48.162043 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:34:48.162043 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:34:48.162043 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:34:48.162043 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:34:48.162043 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:34:48.162043 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:34:48.162043 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:48.162043 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:48.168570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 277/535] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:48.169508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ad126-e0d6-4e7c-9174-4ab3a860c070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:48.170485 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ad126-e0d6-4e7c-9174-4ab3a860c070 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:48.171026 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ad126-e0d6-4e7c-9174-4ab3a860c070 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:48.171498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ad126-e0d6-4e7c-9174-4ab3a860c070 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:48.172504 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ad126-e0d6-4e7c-9174-4ab3a860c070 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:48.173014 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ad126-e0d6-4e7c-9174-4ab3a860c070 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:48.174725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ad126-e0d6-4e7c-9174-4ab3a860c070 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:48.175879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ded5a8-4afd-40f3-b4f7-e09fba7f0174 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:48.206343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 259/536] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:48.210677 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ad126-e0d6-4e7c-9174-4ab3a860c070 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:48.299229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:48.320807 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ded5a8-4afd-40f3-b4f7-e09fba7f0174 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:48.406775 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ad126-e0d6-4e7c-9174-4ab3a860c070 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:48.408248 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c8f4a-c3cc-4cf5-adf6-e78a27526e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:48.409085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c8f4a-c3cc-4cf5-adf6-e78a27526e97 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:48.409550 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c8f4a-c3cc-4cf5-adf6-e78a27526e97 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:48.409978 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c8f4a-c3cc-4cf5-adf6-e78a27526e97 None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:48.411021 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c8f4a-c3cc-4cf5-adf6-e78a27526e97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:48.411480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c8f4a-c3cc-4cf5-adf6-e78a27526e97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:48.413090 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c8f4a-c3cc-4cf5-adf6-e78a27526e97 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:48.433549 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 278/537] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:34:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:48.450486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c8f4a-c3cc-4cf5-adf6-e78a27526e97 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:48.450953 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c36a9-282c-4035-a0e7-5c1d023eda37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:48.451860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c36a9-282c-4035-a0e7-5c1d023eda37 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:48.452235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c36a9-282c-4035-a0e7-5c1d023eda37 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:48.452707 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c36a9-282c-4035-a0e7-5c1d023eda37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:48.460928 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c36a9-282c-4035-a0e7-5c1d023eda37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:48.463550 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c36a9-282c-4035-a0e7-5c1d023eda37 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:48.481401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-906c36a9-282c-4035-a0e7-5c1d023eda37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:48.481401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:48.481401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:48.481401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:48.481401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:48.481401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:48.481401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:48.481401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:48.481401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:48.481401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:48.481401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:48.481401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:48.481401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:34:48.481401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:34:48.481401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:34:48.482505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:34:48.482505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:48.482505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:48.482505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:48.482505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:48.482505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:48.482505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:48.482505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:48.482505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:48.482505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:48.482505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:48.482505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:48.482505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:48.482505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:48.482505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:48.482505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:48.482505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:48.483546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:34:48.483546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:34:48.483546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:34:48.483546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:34:48.483546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:34:48.483546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:34:48.483546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:48.483546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:48.485392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 260/538] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:48.486002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:48.496257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c36a9-282c-4035-a0e7-5c1d023eda37 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:48.608753 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c8f4a-c3cc-4cf5-adf6-e78a27526e97 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:48.645438 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a933b2-b3f5-4628-a8ab-b59fcdd97535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:48.646202 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a933b2-b3f5-4628-a8ab-b59fcdd97535 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:48.646466 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a933b2-b3f5-4628-a8ab-b59fcdd97535 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:48.646743 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a933b2-b3f5-4628-a8ab-b59fcdd97535 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:48.647646 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a933b2-b3f5-4628-a8ab-b59fcdd97535 None admin] RBAC: Adding query filter params (role.id=805851682adf4debbe99b1681c9fcdfc, scope.project.id=7be83fb6f20341139147373837320148, user.id=493fcde4a45944efbc0b52c44e61b411) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:48.647959 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a933b2-b3f5-4628-a8ab-b59fcdd97535 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:48.649537 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a933b2-b3f5-4628-a8ab-b59fcdd97535 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:48.656370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 279/539] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:34:48 2022] GET /identity/v3/role_assignments?user.id=493fcde4a45944efbc0b52c44e61b411&scope.project.id=7be83fb6f20341139147373837320148&role.id=805851682adf4debbe99b1681c9fcdfc => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:48.666467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02a966-808f-4cf2-9444-d3bc71456f2d None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:48.666776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02a966-808f-4cf2-9444-d3bc71456f2d None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:48.667031 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02a966-808f-4cf2-9444-d3bc71456f2d None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:48.668664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 280/540] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:48.684054 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4845c54-07fb-4378-a5f2-8d2121d823fa None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:48.684414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4845c54-07fb-4378-a5f2-8d2121d823fa None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:48.684696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4845c54-07fb-4378-a5f2-8d2121d823fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:48.691369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:48.712696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-e4845c54-07fb-4378-a5f2-8d2121d823fa None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:48.715308 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e4845c54-07fb-4378-a5f2-8d2121d823fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:48.764748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c36a9-282c-4035-a0e7-5c1d023eda37 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:48.810931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 281/541] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:34:48.836450 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4845c54-07fb-4378-a5f2-8d2121d823fa None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:48.837059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a11d9-92da-4f70-b593-fcc8c7850a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:48.838518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a11d9-92da-4f70-b593-fcc8c7850a04 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:48.839007 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a11d9-92da-4f70-b593-fcc8c7850a04 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:48.839566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a11d9-92da-4f70-b593-fcc8c7850a04 None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:48.840729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a11d9-92da-4f70-b593-fcc8c7850a04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:48.841256 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a11d9-92da-4f70-b593-fcc8c7850a04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:48.842979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a11d9-92da-4f70-b593-fcc8c7850a04 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:48.870490 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 261/542] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:34:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:48.875522 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a11d9-92da-4f70-b593-fcc8c7850a04 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:48.946126 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:48.954869 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4845c54-07fb-4378-a5f2-8d2121d823fa None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:48.995093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ad50c-ccff-4a67-af04-ea52cd8c5ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:48.995936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ad50c-ccff-4a67-af04-ea52cd8c5ba8 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:48.996224 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ad50c-ccff-4a67-af04-ea52cd8c5ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:48.996497 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ad50c-ccff-4a67-af04-ea52cd8c5ba8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:48.998615 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:49.005336 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a11d9-92da-4f70-b593-fcc8c7850a04 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:49.017439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ad50c-ccff-4a67-af04-ea52cd8c5ba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:49.038072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-f59ad50c-ccff-4a67-af04-ea52cd8c5ba8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:49.038072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:49.038072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:49.038072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:49.038072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:49.038072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:49.038072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:49.038072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:49.038072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:49.038072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:49.038072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:49.038072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:49.038072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:49.038072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:49.038072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:49.038072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:49.039304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:49.039304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:49.039304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:49.039304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:49.039304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:49.039304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:49.039304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:49.039304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:49.039304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:49.039304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:49.039304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:49.039304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:49.039304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:49.039304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:49.039304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:49.039304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:49.041200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:49.041200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:49.041200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:49.041200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:49.041200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:49.041200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:49.041200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:49.041200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:49.041200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:49.041200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:49.041200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:49.041200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:49.041200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 282/543] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:34:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:49.046442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adef60-37ea-4848-8ab0-4b443a5f1abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:49.046807 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ad50c-ccff-4a67-af04-ea52cd8c5ba8 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:49.047690 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adef60-37ea-4848-8ab0-4b443a5f1abb None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:49.048203 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adef60-37ea-4848-8ab0-4b443a5f1abb None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:49.048672 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adef60-37ea-4848-8ab0-4b443a5f1abb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:49.049782 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07adef60-37ea-4848-8ab0-4b443a5f1abb None admin] RBAC: Adding query filter params (role.id=805851682adf4debbe99b1681c9fcdfc, scope.project.id=7be83fb6f20341139147373837320148, user.id=42dee639e2834d99b7468f911c37cf8f) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:49.050311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07adef60-37ea-4848-8ab0-4b443a5f1abb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:49.052015 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07adef60-37ea-4848-8ab0-4b443a5f1abb None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:49.059970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 262/544] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:34:48 2022] GET /identity/v3/role_assignments?user.id=42dee639e2834d99b7468f911c37cf8f&scope.project.id=7be83fb6f20341139147373837320148&role.id=805851682adf4debbe99b1681c9fcdfc => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:49.173747 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c0729-8352-4fb9-b930-fa7cc6b82476 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:49.173747 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c0729-8352-4fb9-b930-fa7cc6b82476 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:49.173747 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c0729-8352-4fb9-b930-fa7cc6b82476 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:49.174289 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 263/545] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:34:49.179743 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b0833-e702-45d6-b273-ab3748499810 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:49.180062 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b0833-e702-45d6-b273-ab3748499810 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:49.180317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b0833-e702-45d6-b273-ab3748499810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:49.206683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ad50c-ccff-4a67-af04-ea52cd8c5ba8 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:49.223392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-b34b0833-e702-45d6-b273-ab3748499810 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:49.224624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-b34b0833-e702-45d6-b273-ab3748499810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:49.268463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555cb59-35c8-4b18-8381-49452424fc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:49.269153 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555cb59-35c8-4b18-8381-49452424fc28 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:49.269452 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555cb59-35c8-4b18-8381-49452424fc28 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:49.269800 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555cb59-35c8-4b18-8381-49452424fc28 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:49.270515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555cb59-35c8-4b18-8381-49452424fc28 None admin] RBAC: Adding query filter params (name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:49.270823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555cb59-35c8-4b18-8381-49452424fc28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:49.272766 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555cb59-35c8-4b18-8381-49452424fc28 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:49.281505 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 283/546] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:34:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:49.287148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555cb59-35c8-4b18-8381-49452424fc28 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:49.332055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 264/547] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:34:49.350787 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:49.368109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b0833-e702-45d6-b273-ab3748499810 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:49.424525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555cb59-35c8-4b18-8381-49452424fc28 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:49.475689 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:49.503809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc25f1-6caa-47ce-b507-8f68aabf161a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:49.504618 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc25f1-6caa-47ce-b507-8f68aabf161a None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:49.504903 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc25f1-6caa-47ce-b507-8f68aabf161a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:49.505178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc25f1-6caa-47ce-b507-8f68aabf161a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:49.510702 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b0833-e702-45d6-b273-ab3748499810 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:49.531937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc25f1-6caa-47ce-b507-8f68aabf161a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:49.533411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc25f1-6caa-47ce-b507-8f68aabf161a None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:49.569616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-a8fc25f1-6caa-47ce-b507-8f68aabf161a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:49.569616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:49.569616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:49.569616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:49.569616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:49.569616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:49.569616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:49.569616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:49.569616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:49.569616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:49.569616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:49.569616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:49.569616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:49.569616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:49.569616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:49.570538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:49.570538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:49.570538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:49.570538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:49.570538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:49.570538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:49.570538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:49.570538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:49.570538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:49.570538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:49.570538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:49.570538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:49.570538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:49.570538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:49.570538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:49.570538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:49.570538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:49.571995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:49.571995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:49.571995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:49.571995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:49.575142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 284/548] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:34:49.578942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc25f1-6caa-47ce-b507-8f68aabf161a None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:49.583834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ca2e7-7073-4ea4-a0ce-b2d9d80f1f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:49.584920 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ca2e7-7073-4ea4-a0ce-b2d9d80f1f87 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:49.585199 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ca2e7-7073-4ea4-a0ce-b2d9d80f1f87 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:49.585479 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ca2e7-7073-4ea4-a0ce-b2d9d80f1f87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:49.609067 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ca2e7-7073-4ea4-a0ce-b2d9d80f1f87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:49.618507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-eb4ca2e7-7073-4ea4-a0ce-b2d9d80f1f87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:49.618507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:49.618507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:49.618507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:49.618507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:49.618507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:49.618507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:49.618507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:49.618507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:49.618507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:49.618507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:49.618507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:49.618507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:49.618507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:49.618507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:49.618507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:49.619431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:49.619431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:49.619431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:49.619431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:49.619431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:49.619431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:49.619431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:49.619431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:49.619431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:49.619431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:49.619431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:49.619431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:49.619431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:49.619431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:49.619431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:49.619431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:49.620739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:49.620739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:49.620739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:49.620739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:49.620739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:49.620739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:49.620739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:49.620739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:49.620739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:49.620739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:49.620739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:49.620739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:49.620739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 265/549] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:34:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:34:49.630283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ca2e7-7073-4ea4-a0ce-b2d9d80f1f87 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:49.791530 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc25f1-6caa-47ce-b507-8f68aabf161a None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:49.813661 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:49.832739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ca2e7-7073-4ea4-a0ce-b2d9d80f1f87 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:49.863581 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:49.866847 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581e846-d585-449c-bb1b-0999d3c25beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:49.869774 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581e846-d585-449c-bb1b-0999d3c25beb None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:49.870066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581e846-d585-449c-bb1b-0999d3c25beb None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:49.870348 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581e846-d585-449c-bb1b-0999d3c25beb None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:49.871093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581e846-d585-449c-bb1b-0999d3c25beb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:49.871404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581e846-d585-449c-bb1b-0999d3c25beb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:49.877007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581e846-d585-449c-bb1b-0999d3c25beb None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:49.902580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 285/550] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:49.904979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ed191-ac05-401e-a14b-3758cf54c98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:49.905682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ed191-ac05-401e-a14b-3758cf54c98f None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:49.905974 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ed191-ac05-401e-a14b-3758cf54c98f None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:49.906254 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ed191-ac05-401e-a14b-3758cf54c98f None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:49.907018 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ed191-ac05-401e-a14b-3758cf54c98f None admin] RBAC: Adding query filter params (name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:49.907328 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ed191-ac05-401e-a14b-3758cf54c98f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:49.909726 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581e846-d585-449c-bb1b-0999d3c25beb None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:49.910568 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ed191-ac05-401e-a14b-3758cf54c98f None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:49.931201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 266/551] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:34:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:49.937376 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ed191-ac05-401e-a14b-3758cf54c98f None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:50.096073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ed191-ac05-401e-a14b-3758cf54c98f None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:50.114958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581e846-d585-449c-bb1b-0999d3c25beb None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:50.138745 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c307814-877e-4de8-8b7d-a098ee2e6cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:50.139523 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c307814-877e-4de8-8b7d-a098ee2e6cd4 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.139802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c307814-877e-4de8-8b7d-a098ee2e6cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.140075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c307814-877e-4de8-8b7d-a098ee2e6cd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.151622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c307814-877e-4de8-8b7d-a098ee2e6cd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:50.152937 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c307814-877e-4de8-8b7d-a098ee2e6cd4 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:50.161073 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12febf-60ce-4217-be94-0e8c252c6f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:50.162859 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12febf-60ce-4217-be94-0e8c252c6f6c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.163331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12febf-60ce-4217-be94-0e8c252c6f6c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.164038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12febf-60ce-4217-be94-0e8c252c6f6c None admin] PATH_INFO: `/v3/users/swift` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.171844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-2c307814-877e-4de8-8b7d-a098ee2e6cd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:50.171844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:50.171844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:50.171844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:50.171844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:50.171844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:50.171844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:50.171844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:50.171844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:50.171844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:50.171844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:50.171844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:50.171844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:50.171844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:50.171844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:50.172812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:50.172812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:50.172812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:50.172812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:50.172812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:50.172812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:50.172812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:50.172812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:50.172812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:50.172812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:50.172812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:50.172812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:50.172812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:50.172812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:50.172812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:50.172812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:50.172812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:50.173701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:50.173701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:50.173701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:50.173701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:50.173701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 267/552] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:50.175114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32779c8c-2df4-44d1-a9e3-4941663e67d3 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.175338 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12febf-60ce-4217-be94-0e8c252c6f6c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:50.175437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32779c8c-2df4-44d1-a9e3-4941663e67d3 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.175788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32779c8c-2df4-44d1-a9e3-4941663e67d3 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.177249 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12febf-60ce-4217-be94-0e8c252c6f6c None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:50.182059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 268/553] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:50.183569 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32779c8c-2df4-44d1-a9e3-4941663e67d3 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:50.197524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-6b12febf-60ce-4217-be94-0e8c252c6f6c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:34:50.197524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:50.197524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:50.197524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:50.197524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:50.197524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:50.197524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:50.197524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:50.197524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:50.197524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:50.197524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:50.197524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:50.197524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:34:50.197524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:34:50.197524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:34:50.197524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:34:50.198506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:50.198506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:50.198506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:34:50.198506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:50.198506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:34:50.198506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:50.198506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:50.198506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:50.198506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:50.198506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:50.198506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:50.198506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:50.198506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:50.198506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:50.198506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:50.198506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:50.198506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:50.198506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:50.199398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:50.199398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:50.199398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:34:50.199398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:34:50.199398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:34:50.199398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:34:50.199398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:34:50.199398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:34:50.199398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:34:50.199398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:50.202475 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 286/554] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:34:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:34:50.207497 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a47a12-5e8b-4755-918b-ffaa96240f2b None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.207497 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a47a12-5e8b-4755-918b-ffaa96240f2b None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.207905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a47a12-5e8b-4755-918b-ffaa96240f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.225987 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:50.234505 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-35a47a12-5e8b-4755-918b-ffaa96240f2b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:50.237143 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-35a47a12-5e8b-4755-918b-ffaa96240f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:50.268967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:50.313081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32779c8c-2df4-44d1-a9e3-4941663e67d3 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:50.338483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 287/555] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:50.339991 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a47a12-5e8b-4755-918b-ffaa96240f2b None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:50.366505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2dbb31-ac6a-4c9b-9978-88a12d4eb431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:50.367311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dbb31-ac6a-4c9b-9978-88a12d4eb431 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.367446 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dbb31-ac6a-4c9b-9978-88a12d4eb431 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.367699 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dbb31-ac6a-4c9b-9978-88a12d4eb431 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.368565 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2dbb31-ac6a-4c9b-9978-88a12d4eb431 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:50.369187 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2dbb31-ac6a-4c9b-9978-88a12d4eb431 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:50.370656 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2dbb31-ac6a-4c9b-9978-88a12d4eb431 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:50.389981 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 269/556] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:50.391063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2dbb31-ac6a-4c9b-9978-88a12d4eb431 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:50.456367 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a47a12-5e8b-4755-918b-ffaa96240f2b None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:50.494895 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67f18a-05a9-45d8-8e55-ae6d6e50d698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:50.495978 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67f18a-05a9-45d8-8e55-ae6d6e50d698 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.496489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67f18a-05a9-45d8-8e55-ae6d6e50d698 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.496963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67f18a-05a9-45d8-8e55-ae6d6e50d698 None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.498060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67f18a-05a9-45d8-8e55-ae6d6e50d698 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:50.498566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:50.499678 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67f18a-05a9-45d8-8e55-ae6d6e50d698 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:50.501565 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67f18a-05a9-45d8-8e55-ae6d6e50d698 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:50.515805 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2dbb31-ac6a-4c9b-9978-88a12d4eb431 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:50.526593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 288/557] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Jun 27 15:34:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:50.527726 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67f18a-05a9-45d8-8e55-ae6d6e50d698 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:50.554314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:50.567980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c58f2-00e8-4fcd-a9dc-d8227e808d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:50.569287 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c58f2-00e8-4fcd-a9dc-d8227e808d7f None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.569965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c58f2-00e8-4fcd-a9dc-d8227e808d7f None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.570799 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c58f2-00e8-4fcd-a9dc-d8227e808d7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.585736 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c58f2-00e8-4fcd-a9dc-d8227e808d7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:50.597658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-104c58f2-00e8-4fcd-a9dc-d8227e808d7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:50.597658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:50.597658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:50.597658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:50.597658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:50.597658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:50.597658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:50.597658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:50.597658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:50.597658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:50.597658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:50.597658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:50.597658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:50.597658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:50.597658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:50.597658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:50.599323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:50.599323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:50.599323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:50.599323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:50.599323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:50.599323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:50.599323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:50.599323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:50.599323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:50.599323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:50.599323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:50.599323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:50.599323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:50.599323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:50.599323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:50.599323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:50.600343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:50.600343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:50.600343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:50.600343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:50.600343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:50.600343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:50.600343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:50.600343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:50.600343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:50.600343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:50.600343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:50.600343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:50.604072 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 270/558] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:34:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:50.605319 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c58f2-00e8-4fcd-a9dc-d8227e808d7f None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:50.655159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67f18a-05a9-45d8-8e55-ae6d6e50d698 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:50.691821 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9eadf-d906-4537-aeaa-fd7d75c1c241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:50.692806 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9eadf-d906-4537-aeaa-fd7d75c1c241 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.693132 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9eadf-d906-4537-aeaa-fd7d75c1c241 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.693541 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9eadf-d906-4537-aeaa-fd7d75c1c241 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.701561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9eadf-d906-4537-aeaa-fd7d75c1c241 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:50.703205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9eadf-d906-4537-aeaa-fd7d75c1c241 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:50.719406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-a6e9eadf-d906-4537-aeaa-fd7d75c1c241 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:34:50.719406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:50.719406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:50.719406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:50.719406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:50.719406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:50.719406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:50.719406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:50.719406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:50.719406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:50.719406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:50.719406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:50.719406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:34:50.719406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:34:50.719406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:34:50.719406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:34:50.720589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:50.720589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:50.720589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:34:50.720589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:50.720589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:34:50.720589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:50.720589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:50.720589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:50.720589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:50.720589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:50.720589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:50.720589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:50.720589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:50.720589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:50.720589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:50.720589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:50.720589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:50.720589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:50.721902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:50.721902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:50.721902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:34:50.721902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:34:50.721902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:34:50.721902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:34:50.721902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:34:50.721902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:34:50.721902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:34:50.721902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:50.729257 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 289/559] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Jun 27 15:34:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:50.729830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9eadf-d906-4537-aeaa-fd7d75c1c241 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:50.745604 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c58f2-00e8-4fcd-a9dc-d8227e808d7f None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:50.794917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656e98b-9009-4e2d-98a5-a84bcb33f50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:50.796105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656e98b-9009-4e2d-98a5-a84bcb33f50c None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.796576 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656e98b-9009-4e2d-98a5-a84bcb33f50c None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.797250 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656e98b-9009-4e2d-98a5-a84bcb33f50c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.814856 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:50.824367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:50.859922 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9eadf-d906-4537-aeaa-fd7d75c1c241 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:50.894843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656e98b-9009-4e2d-98a5-a84bcb33f50c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:50.897069 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656e98b-9009-4e2d-98a5-a84bcb33f50c None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:50.914122 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e27cfd-7eb3-40b9-a297-b358e605c68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:50.914739 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27cfd-7eb3-40b9-a297-b358e605c68e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:50.915069 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27cfd-7eb3-40b9-a297-b358e605c68e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:50.915605 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27cfd-7eb3-40b9-a297-b358e605c68e None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:50.916517 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e27cfd-7eb3-40b9-a297-b358e605c68e None admin] RBAC: Adding query filter params (name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:50.916831 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e27cfd-7eb3-40b9-a297-b358e605c68e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:50.918956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e27cfd-7eb3-40b9-a297-b358e605c68e None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:50.931469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 290/560] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:34:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:50.932817 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e27cfd-7eb3-40b9-a297-b358e605c68e None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:51.012349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 271/561] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:51.013647 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656e98b-9009-4e2d-98a5-a84bcb33f50c None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:51.067025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:51.075193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e27cfd-7eb3-40b9-a297-b358e605c68e None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:51.112303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa02419-15a0-4ad9-a07c-bc629b7287eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:51.112738 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa02419-15a0-4ad9-a07c-bc629b7287eb None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:51.113025 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa02419-15a0-4ad9-a07c-bc629b7287eb None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:51.113304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa02419-15a0-4ad9-a07c-bc629b7287eb None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:51.114199 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa02419-15a0-4ad9-a07c-bc629b7287eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:51.114604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa02419-15a0-4ad9-a07c-bc629b7287eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:51.116003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa02419-15a0-4ad9-a07c-bc629b7287eb None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:51.127598 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:51.132064 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656e98b-9009-4e2d-98a5-a84bcb33f50c None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:51.144850 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 291/562] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Jun 27 15:34:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:51.145781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa02419-15a0-4ad9-a07c-bc629b7287eb None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:51.177867 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ddb3b-8af9-454c-958c-8108f0f7dd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:51.179051 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ddb3b-8af9-454c-958c-8108f0f7dd30 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:51.179540 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ddb3b-8af9-454c-958c-8108f0f7dd30 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:51.180101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ddb3b-8af9-454c-958c-8108f0f7dd30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:51.194393 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ddb3b-8af9-454c-958c-8108f0f7dd30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:51.195881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ddb3b-8af9-454c-958c-8108f0f7dd30 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:51.215127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-244ddb3b-8af9-454c-958c-8108f0f7dd30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:51.215127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:51.215127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:51.215127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:51.215127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:51.215127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:51.215127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:51.215127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:51.215127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:51.215127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:51.215127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:51.215127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:51.215127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:51.215127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:51.215127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:51.216277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:51.216277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:51.216277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:51.216277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:51.216277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:51.216277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:51.216277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:51.216277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:51.216277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:51.216277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:51.216277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:51.216277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:51.216277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:51.216277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:51.216277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:51.216277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:51.216277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:51.217226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:51.217226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:51.217226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:51.217226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:51.217226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 272/563] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:51.217226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ddb3b-8af9-454c-958c-8108f0f7dd30 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:51.270908 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa02419-15a0-4ad9-a07c-bc629b7287eb None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:51.308474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6eb8b-4c90-482e-890b-f089816f7b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:51.309263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6eb8b-4c90-482e-890b-f089816f7b5d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:51.309542 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6eb8b-4c90-482e-890b-f089816f7b5d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:51.309843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6eb8b-4c90-482e-890b-f089816f7b5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:51.320159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6eb8b-4c90-482e-890b-f089816f7b5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:51.322279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6eb8b-4c90-482e-890b-f089816f7b5d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:51.324589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:51.328566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ddb3b-8af9-454c-958c-8108f0f7dd30 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:51.342467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-16c6eb8b-4c90-482e-890b-f089816f7b5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:51.342467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:51.342467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:51.342467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:51.342467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:51.342467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:51.342467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:51.342467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:51.342467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:51.342467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:51.342467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:51.342467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:51.342467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:51.342467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:51.342467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:51.343393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:51.343393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:51.343393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:51.343393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:51.343393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:51.343393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:51.343393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:51.343393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:51.343393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:51.343393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:51.343393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:51.343393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:51.343393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:51.343393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:51.343393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:51.343393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:51.343393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:51.344298 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:51.344298 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:51.344298 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:51.344298 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:51.344298 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 292/564] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:51.345202 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827b9d9-0c29-41c8-ba7a-d76e961fe66c None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:51.345485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827b9d9-0c29-41c8-ba7a-d76e961fe66c None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:51.345782 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827b9d9-0c29-41c8-ba7a-d76e961fe66c None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:51.349689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 293/565] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:51.349689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827b9d9-0c29-41c8-ba7a-d76e961fe66c None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:51.376123 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cd3e3-3a15-4c1b-b6e7-e08283377a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:51.376930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cd3e3-3a15-4c1b-b6e7-e08283377a8f None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:51.377216 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cd3e3-3a15-4c1b-b6e7-e08283377a8f None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:51.377497 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cd3e3-3a15-4c1b-b6e7-e08283377a8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:51.389067 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81cd3e3-3a15-4c1b-b6e7-e08283377a8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:51.390579 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81cd3e3-3a15-4c1b-b6e7-e08283377a8f None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:51.408901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-c81cd3e3-3a15-4c1b-b6e7-e08283377a8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:51.408901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:51.408901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:51.408901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:51.408901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:51.408901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:51.408901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:51.408901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:51.408901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:51.408901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:51.408901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:51.408901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:51.408901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:51.408901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:51.408901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:51.410156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:51.410156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:51.410156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:51.410156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:51.410156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:51.410156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:51.410156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:51.410156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:51.410156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:51.410156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:51.410156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:51.410156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:51.410156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:51.410156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:51.410156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:51.410156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:51.410156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:51.411084 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:51.411084 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:51.411084 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:51.411084 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:51.411084 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 273/566] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:51.411381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:51.411467 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cd3e3-3a15-4c1b-b6e7-e08283377a8f None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:51.486785 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827b9d9-0c29-41c8-ba7a-d76e961fe66c None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:51.512194 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cd3e3-3a15-4c1b-b6e7-e08283377a8f None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:51.542588 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e197d8-c93e-4ee3-b8ce-872f91e10f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:51.543245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e197d8-c93e-4ee3-b8ce-872f91e10f05 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:51.543535 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e197d8-c93e-4ee3-b8ce-872f91e10f05 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:51.543801 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e197d8-c93e-4ee3-b8ce-872f91e10f05 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:51.544511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e197d8-c93e-4ee3-b8ce-872f91e10f05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:51.544813 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e197d8-c93e-4ee3-b8ce-872f91e10f05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:51.546203 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e197d8-c93e-4ee3-b8ce-872f91e10f05 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:51.547041 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf08d6-c58a-493b-83b0-e6dea618510c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:51.547712 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf08d6-c58a-493b-83b0-e6dea618510c None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:51.547996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf08d6-c58a-493b-83b0-e6dea618510c None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:51.548271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf08d6-c58a-493b-83b0-e6dea618510c None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:51.548997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf08d6-c58a-493b-83b0-e6dea618510c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:51.549304 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf08d6-c58a-493b-83b0-e6dea618510c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:51.550702 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf08d6-c58a-493b-83b0-e6dea618510c None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:51.557575 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 294/567] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:51.558835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a56161-9dfb-4c0e-a0c0-118250d95189 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:51.559397 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a56161-9dfb-4c0e-a0c0-118250d95189 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:51.559397 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a56161-9dfb-4c0e-a0c0-118250d95189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:51.566442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 274/568] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:51.566442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d14a2-de14-4e16-b459-c4a91f84b5e8 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:51.566442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d14a2-de14-4e16-b459-c4a91f84b5e8 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:51.566442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d14a2-de14-4e16-b459-c4a91f84b5e8 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:51.566442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 275/569] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:51.566442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d14a2-de14-4e16-b459-c4a91f84b5e8 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:51.579766 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-53a56161-9dfb-4c0e-a0c0-118250d95189 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:51.580979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-53a56161-9dfb-4c0e-a0c0-118250d95189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:51.581213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:51.652570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 295/570] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:51.653407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a56161-9dfb-4c0e-a0c0-118250d95189 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:51.684015 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d14a2-de14-4e16-b459-c4a91f84b5e8 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:51.695465 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:51.719684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f330dd-62df-4e21-9987-51ed8cf04b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:51.720349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f330dd-62df-4e21-9987-51ed8cf04b95 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:51.720628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f330dd-62df-4e21-9987-51ed8cf04b95 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:51.720899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f330dd-62df-4e21-9987-51ed8cf04b95 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:51.721618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f330dd-62df-4e21-9987-51ed8cf04b95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:51.721966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f330dd-62df-4e21-9987-51ed8cf04b95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:51.723349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f330dd-62df-4e21-9987-51ed8cf04b95 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:51.735184 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 276/571] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:51.735988 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f330dd-62df-4e21-9987-51ed8cf04b95 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:51.761772 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a56161-9dfb-4c0e-a0c0-118250d95189 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:51.796055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a7cef-98aa-4f82-9df9-ab294871cd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:51.796855 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a7cef-98aa-4f82-9df9-ab294871cd1a None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:51.797138 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a7cef-98aa-4f82-9df9-ab294871cd1a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:51.797415 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a7cef-98aa-4f82-9df9-ab294871cd1a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:51.804955 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a7cef-98aa-4f82-9df9-ab294871cd1a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:51.806557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a7cef-98aa-4f82-9df9-ab294871cd1a None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:51.806699 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:51.821204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-f39a7cef-98aa-4f82-9df9-ab294871cd1a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 15:34:51.821204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:51.821204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:51.821204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:51.821204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:51.821204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:51.821204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:51.821204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:51.821204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:51.821204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:51.821204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:51.821204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:51.821204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:34:51.821204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:34:51.821204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:34:51.821204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:34:51.823189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:51.823189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:51.823189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:34:51.823189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:51.823189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:34:51.823189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:51.823189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:51.823189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:51.823189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:51.823189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:51.823189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:51.823189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:51.823189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:51.823189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:51.823189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:51.823189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:51.823189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:51.823189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:51.824135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:51.824135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:51.824135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:34:51.824135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:34:51.824135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:34:51.824135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:34:51.824135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:34:51.824135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:34:51.824135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 15:34:51.824135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:51.824135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 296/572] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:51.824794 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00699800-4bc0-426a-9383-78e493ec4cf9 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:51.824794 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00699800-4bc0-426a-9383-78e493ec4cf9 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:51.824794 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00699800-4bc0-426a-9383-78e493ec4cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:51.842058 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-00699800-4bc0-426a-9383-78e493ec4cf9 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:51.842922 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f330dd-62df-4e21-9987-51ed8cf04b95 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:51.843236 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-00699800-4bc0-426a-9383-78e493ec4cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:51.878840 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e753e1-f9ba-416b-8b51-c5b8d07a8dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:51.879640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e753e1-f9ba-416b-8b51-c5b8d07a8dee None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:51.879921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e753e1-f9ba-416b-8b51-c5b8d07a8dee None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:51.880206 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e753e1-f9ba-416b-8b51-c5b8d07a8dee None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:51.887142 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e753e1-f9ba-416b-8b51-c5b8d07a8dee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:51.888598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e753e1-f9ba-416b-8b51-c5b8d07a8dee None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:51.904048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-65e753e1-f9ba-416b-8b51-c5b8d07a8dee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:51.904048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:51.904048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:51.904048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:51.904048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:51.904048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:51.904048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:51.904048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:51.904048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:51.904048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:51.904048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:51.904048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:51.904048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:34:51.904048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:34:51.904048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:34:51.905431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:34:51.905431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:51.905431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:51.905431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:51.905431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:51.905431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:51.905431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:51.905431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:51.905431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:51.905431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:51.905431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:51.905431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:51.905431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:51.905431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:51.905431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:51.905431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:51.905431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:51.906650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:34:51.906650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:34:51.906650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:34:51.906650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:34:51.906650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:34:51.906650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:34:51.906650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:51.906650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:51.906650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 277/573] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:51.906650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e753e1-f9ba-416b-8b51-c5b8d07a8dee None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:51.918704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 297/574] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:51.919543 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00699800-4bc0-426a-9383-78e493ec4cf9 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:51.948329 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:52.021183 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00699800-4bc0-426a-9383-78e493ec4cf9 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:52.025496 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e753e1-f9ba-416b-8b51-c5b8d07a8dee None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:52.054967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1d1d9-8866-4e1c-8503-d1824c67b400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:52.055325 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:52.055783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1d1d9-8866-4e1c-8503-d1824c67b400 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:52.056231 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1d1d9-8866-4e1c-8503-d1824c67b400 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:52.056791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1d1d9-8866-4e1c-8503-d1824c67b400 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:52.060901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302589ed-6067-46b9-a629-ad4672afa799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:52.061719 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302589ed-6067-46b9-a629-ad4672afa799 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:52.062004 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302589ed-6067-46b9-a629-ad4672afa799 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:52.062276 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302589ed-6067-46b9-a629-ad4672afa799 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:52.063693 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1d1d9-8866-4e1c-8503-d1824c67b400 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:52.065136 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1d1d9-8866-4e1c-8503-d1824c67b400 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:52.072953 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302589ed-6067-46b9-a629-ad4672afa799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:52.079189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-7dc1d1d9-8866-4e1c-8503-d1824c67b400 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:52.079189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:52.079189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:52.079189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:52.079189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:52.079189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:52.079189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:52.079189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:52.079189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:52.079189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:52.079189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:52.079189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:52.079189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:34:52.079189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:34:52.079189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:34:52.080067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:34:52.080067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:52.080067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:52.080067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:52.080067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:52.080067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:52.080067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:52.080067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:52.080067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:52.080067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:52.080067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:52.080067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:52.080067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:52.080067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:52.080067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:52.080067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:52.080067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:52.080928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:34:52.080928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:34:52.080928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:34:52.080928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:34:52.080928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:34:52.080928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:34:52.080928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:52.080928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:52.081368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-302589ed-6067-46b9-a629-ad4672afa799 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:52.081368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:52.081368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:52.081368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:52.081368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:52.081368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:52.081368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:52.081368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:52.081368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:52.081368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:52.081368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:52.081368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:52.081368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:52.081368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:52.081368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:52.081368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:52.088858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:52.088858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:52.088858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:52.088858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:52.088858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:52.088858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:52.088858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:52.088858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:52.088858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:52.088858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:52.088858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:52.088858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:52.088858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:52.088858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:52.088858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:52.088858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:52.089916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:52.089916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:52.089916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:52.089916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:52.089916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:52.089916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:52.089916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:52.089916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:52.089916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:52.089916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:52.089916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:52.089916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:52.090928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 298/575] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:52.090928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1d1d9-8866-4e1c-8503-d1824c67b400 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:52.091096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 278/576] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:34:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:52.091096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302589ed-6067-46b9-a629-ad4672afa799 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:52.172831 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:52.185138 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1d1d9-8866-4e1c-8503-d1824c67b400 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:52.197621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302589ed-6067-46b9-a629-ad4672afa799 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:52.220936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054742b3-4e04-4443-890f-d6a9de97c0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:52.221613 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054742b3-4e04-4443-890f-d6a9de97c0c7 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:52.221926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054742b3-4e04-4443-890f-d6a9de97c0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:52.222202 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054742b3-4e04-4443-890f-d6a9de97c0c7 None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:52.223040 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054742b3-4e04-4443-890f-d6a9de97c0c7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:52.223349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054742b3-4e04-4443-890f-d6a9de97c0c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:52.224779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054742b3-4e04-4443-890f-d6a9de97c0c7 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:52.232729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d43e39-843b-4848-aa3b-0761b31e4fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:52.233399 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d43e39-843b-4848-aa3b-0761b31e4fcf None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:52.233717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d43e39-843b-4848-aa3b-0761b31e4fcf None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:52.234001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d43e39-843b-4848-aa3b-0761b31e4fcf None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:52.234801 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d43e39-843b-4848-aa3b-0761b31e4fcf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:52.235114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d43e39-843b-4848-aa3b-0761b31e4fcf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:52.236545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d43e39-843b-4848-aa3b-0761b31e4fcf None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:52.242655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 299/577] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Jun 27 15:34:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:52.243794 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054742b3-4e04-4443-890f-d6a9de97c0c7 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:52.250243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 279/578] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:34:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:52.251378 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d43e39-843b-4848-aa3b-0761b31e4fcf None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:52.307327 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:52.348949 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054742b3-4e04-4443-890f-d6a9de97c0c7 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:52.358625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d43e39-843b-4848-aa3b-0761b31e4fcf None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:52.382533 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7efcd-89a9-41ae-9755-8c428e751a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:52.383325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7efcd-89a9-41ae-9755-8c428e751a49 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:52.383610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7efcd-89a9-41ae-9755-8c428e751a49 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:52.384376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7efcd-89a9-41ae-9755-8c428e751a49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:52.393057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8544971-1262-4e94-86fe-8f53f96d1513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:52.393762 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8544971-1262-4e94-86fe-8f53f96d1513 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:52.394046 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8544971-1262-4e94-86fe-8f53f96d1513 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:52.394325 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8544971-1262-4e94-86fe-8f53f96d1513 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:52.394515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7efcd-89a9-41ae-9755-8c428e751a49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:52.395071 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8544971-1262-4e94-86fe-8f53f96d1513 None admin] RBAC: Adding query filter params (name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:52.395380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8544971-1262-4e94-86fe-8f53f96d1513 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:52.396820 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8544971-1262-4e94-86fe-8f53f96d1513 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:52.402882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-89d7efcd-89a9-41ae-9755-8c428e751a49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:52.402882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:52.402882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:52.402882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:52.402882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:52.402882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:52.402882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:52.402882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:52.402882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:52.402882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:52.402882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:52.402882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:52.402882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:52.402882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:52.402882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:52.402882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:52.403820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:52.403820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:52.403820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:52.403820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:52.403820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:52.403820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:52.403820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:52.403820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:52.403820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:52.403820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:52.403820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:52.403820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:52.403820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:52.403820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:52.403820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:52.403820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:52.404645 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:52.404645 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:52.404645 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:52.404645 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:52.404645 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:52.404645 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:52.404645 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:52.404645 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:52.404645 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:52.404645 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:52.404645 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:52.404645 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:52.404645 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 300/579] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:34:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:52.405344 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7efcd-89a9-41ae-9755-8c428e751a49 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:52.407623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 280/580] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:34:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:52.408467 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8544971-1262-4e94-86fe-8f53f96d1513 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:52.419081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:52.514769 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7efcd-89a9-41ae-9755-8c428e751a49 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:52.515391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8544971-1262-4e94-86fe-8f53f96d1513 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:52.529934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:52.550115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf7c6f-d76c-4210-93df-caa85262b7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:52.550773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf7c6f-d76c-4210-93df-caa85262b7d9 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:52.551052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf7c6f-d76c-4210-93df-caa85262b7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:52.551324 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf7c6f-d76c-4210-93df-caa85262b7d9 None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:52.552113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf7c6f-d76c-4210-93df-caa85262b7d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:52.552418 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf7c6f-d76c-4210-93df-caa85262b7d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:52.552670 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17abf2-73fe-4c6a-882c-4ada06d56e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:52.553471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17abf2-73fe-4c6a-882c-4ada06d56e37 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:52.553799 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17abf2-73fe-4c6a-882c-4ada06d56e37 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:52.553901 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf7c6f-d76c-4210-93df-caa85262b7d9 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:52.554078 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17abf2-73fe-4c6a-882c-4ada06d56e37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:52.564839 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17abf2-73fe-4c6a-882c-4ada06d56e37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:52.565732 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 301/581] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:34:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:52.566258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17abf2-73fe-4c6a-882c-4ada06d56e37 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:52.566519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf7c6f-d76c-4210-93df-caa85262b7d9 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:52.588501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-0e17abf2-73fe-4c6a-882c-4ada06d56e37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:52.588501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:52.588501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:52.588501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:52.588501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:52.588501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:52.588501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:52.588501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:52.588501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:52.588501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:52.588501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:52.588501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:52.588501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:52.588501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:52.588501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:52.589429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:52.589429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:52.589429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:52.589429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:52.589429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:52.589429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:52.589429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:52.589429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:52.589429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:52.589429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:52.589429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:52.589429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:52.589429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:52.589429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:52.589429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:52.589429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:52.589429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:52.590348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:52.590348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:52.590348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:52.590348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:52.591008 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 281/582] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:52.591992 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17abf2-73fe-4c6a-882c-4ada06d56e37 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:52.637024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:52.674981 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf7c6f-d76c-4210-93df-caa85262b7d9 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:52.691323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17abf2-73fe-4c6a-882c-4ada06d56e37 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:52.709046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d890e5f-8598-4e70-a79e-76fdf9ebecc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:52.709735 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d890e5f-8598-4e70-a79e-76fdf9ebecc2 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:52.710023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d890e5f-8598-4e70-a79e-76fdf9ebecc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:52.710297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d890e5f-8598-4e70-a79e-76fdf9ebecc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:52.711122 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d890e5f-8598-4e70-a79e-76fdf9ebecc2 None admin] RBAC: Adding query filter params (role.id=805851682adf4debbe99b1681c9fcdfc, scope.project.id=7be83fb6f20341139147373837320148, user.id=8e83ea69ded040448fe75caa221ad377) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:52.711435 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d890e5f-8598-4e70-a79e-76fdf9ebecc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:52.712995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d890e5f-8598-4e70-a79e-76fdf9ebecc2 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:52.718372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 302/583] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:34:52 2022] GET /identity/v3/role_assignments?user.id=8e83ea69ded040448fe75caa221ad377&scope.project.id=7be83fb6f20341139147373837320148&role.id=805851682adf4debbe99b1681c9fcdfc => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:52.719155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d890e5f-8598-4e70-a79e-76fdf9ebecc2 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:52.727265 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca21d4-0426-4e76-bd9e-75edc9a9a890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:52.728931 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca21d4-0426-4e76-bd9e-75edc9a9a890 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:52.729220 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca21d4-0426-4e76-bd9e-75edc9a9a890 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:52.729498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca21d4-0426-4e76-bd9e-75edc9a9a890 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:52.730321 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca21d4-0426-4e76-bd9e-75edc9a9a890 None admin] RBAC: Adding query filter params (name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:52.730628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca21d4-0426-4e76-bd9e-75edc9a9a890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:52.732625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca21d4-0426-4e76-bd9e-75edc9a9a890 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:52.747741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 282/584] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:34:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:52.747741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca21d4-0426-4e76-bd9e-75edc9a9a890 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:52.778757 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:52.852629 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d890e5f-8598-4e70-a79e-76fdf9ebecc2 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:52.868780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca21d4-0426-4e76-bd9e-75edc9a9a890 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:52.884078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:52.896081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e18a0a-c3b6-483b-8fda-74305a0e3bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:52.896892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e18a0a-c3b6-483b-8fda-74305a0e3bef None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:52.897172 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e18a0a-c3b6-483b-8fda-74305a0e3bef None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:52.897436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e18a0a-c3b6-483b-8fda-74305a0e3bef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:52.910060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e18a0a-c3b6-483b-8fda-74305a0e3bef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:52.910251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725440c-9611-4685-bb21-ea60ec03ea48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:52.910958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725440c-9611-4685-bb21-ea60ec03ea48 None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:52.911243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725440c-9611-4685-bb21-ea60ec03ea48 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:52.911519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1725440c-9611-4685-bb21-ea60ec03ea48 None admin] PATH_INFO: `/v3/projects/7be83fb6f20341139147373837320148/users/1c8f6262cf24419daf803ae71aed24f5/roles/805851682adf4debbe99b1681c9fcdfc` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:52.911619 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e18a0a-c3b6-483b-8fda-74305a0e3bef None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:52.927544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1725440c-9611-4685-bb21-ea60ec03ea48 None admin] RBAC: Authorizing `identity:create_grant(project_id=7be83fb6f20341139147373837320148, user_id=1c8f6262cf24419daf803ae71aed24f5, role_id=805851682adf4debbe99b1681c9fcdfc)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:52.929001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1725440c-9611-4685-bb21-ea60ec03ea48 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:52.932744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-54e18a0a-c3b6-483b-8fda-74305a0e3bef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:52.932744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:52.932744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:52.932744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:52.932744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:52.932744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:52.932744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:52.932744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:52.932744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:52.932744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:52.932744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:52.932744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:52.932744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:52.932744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:52.932744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:52.933585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:52.933585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:52.933585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:52.933585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:52.933585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:52.933585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:52.933585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:52.933585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:52.933585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:52.933585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:52.933585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:52.933585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:52.933585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:52.933585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:52.933585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:52.933585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:52.933585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:52.939402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:52.939402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:52.939402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:52.939402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:52.939402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 303/585] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:52.939402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e18a0a-c3b6-483b-8fda-74305a0e3bef None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:52.961835 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 283/586] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:34:52 2022] PUT /identity/v3/projects/7be83fb6f20341139147373837320148/users/1c8f6262cf24419daf803ae71aed24f5/roles/805851682adf4debbe99b1681c9fcdfc => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:52.961835 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725440c-9611-4685-bb21-ea60ec03ea48 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:53.057538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e18a0a-c3b6-483b-8fda-74305a0e3bef None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:53.079557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:53.099104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b519b-63e2-430e-acf5-b2a88079db9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:53.099830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b519b-63e2-430e-acf5-b2a88079db9d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:53.100111 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b519b-63e2-430e-acf5-b2a88079db9d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:53.100421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b519b-63e2-430e-acf5-b2a88079db9d None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:53.101163 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b519b-63e2-430e-acf5-b2a88079db9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:53.101468 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b519b-63e2-430e-acf5-b2a88079db9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:53.103546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b519b-63e2-430e-acf5-b2a88079db9d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:53.111178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1725440c-9611-4685-bb21-ea60ec03ea48 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:53.117727 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 304/587] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:53.118653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b519b-63e2-430e-acf5-b2a88079db9d None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:53.159936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50a255-944f-4dbd-8fe4-96a03d733e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:53.161022 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50a255-944f-4dbd-8fe4-96a03d733e87 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:53.161466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50a255-944f-4dbd-8fe4-96a03d733e87 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:53.161939 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50a255-944f-4dbd-8fe4-96a03d733e87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:53.173670 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50a255-944f-4dbd-8fe4-96a03d733e87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:53.175178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50a255-944f-4dbd-8fe4-96a03d733e87 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:53.177461 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:53.199675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-6b50a255-944f-4dbd-8fe4-96a03d733e87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:53.199675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:53.199675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:53.199675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:53.199675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:53.199675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:53.199675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:53.199675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:53.199675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:53.199675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:53.199675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:53.199675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:53.199675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:53.199675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:53.199675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:53.200605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:53.200605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:53.200605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:53.200605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:53.200605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:53.200605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:53.200605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:53.200605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:53.200605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:53.200605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:53.200605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:53.200605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:53.200605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:53.200605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:53.200605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:53.200605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:53.200605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:53.201512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:53.201512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:53.201512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:53.201512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:53.204084 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 284/588] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:53.204989 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50a255-944f-4dbd-8fe4-96a03d733e87 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:53.248232 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b519b-63e2-430e-acf5-b2a88079db9d None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:53.292783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e8ed5-e759-4e36-9c05-fc4bae7ff8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:53.294406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e8ed5-e759-4e36-9c05-fc4bae7ff8db None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:53.294901 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e8ed5-e759-4e36-9c05-fc4bae7ff8db None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:53.295659 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e8ed5-e759-4e36-9c05-fc4bae7ff8db None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:53.296573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e8ed5-e759-4e36-9c05-fc4bae7ff8db None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:53.297126 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e8ed5-e759-4e36-9c05-fc4bae7ff8db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:53.299730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e8ed5-e759-4e36-9c05-fc4bae7ff8db None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:53.315401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 305/589] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:53.316160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e8ed5-e759-4e36-9c05-fc4bae7ff8db None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:53.348394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50a255-944f-4dbd-8fe4-96a03d733e87 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:53.389294 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:53.398891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f016d61-3578-4da7-9b67-8ee52b23fa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:53.400706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f016d61-3578-4da7-9b67-8ee52b23fa16 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:53.401267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f016d61-3578-4da7-9b67-8ee52b23fa16 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:53.401780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f016d61-3578-4da7-9b67-8ee52b23fa16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:53.411659 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f016d61-3578-4da7-9b67-8ee52b23fa16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:53.414758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f016d61-3578-4da7-9b67-8ee52b23fa16 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:53.432162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-1f016d61-3578-4da7-9b67-8ee52b23fa16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:53.432162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:53.432162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:53.432162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:53.432162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:53.432162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:53.432162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:53.432162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:53.432162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:53.432162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:53.432162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:53.432162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:53.432162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:34:53.432162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:34:53.432162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:34:53.433198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:34:53.433198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:53.433198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:53.433198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:53.433198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:53.433198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:53.433198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:53.433198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:53.433198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:53.433198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:53.433198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:53.433198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:53.433198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:53.433198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:53.433198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:53.433198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:53.433198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:53.434109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:34:53.434109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:34:53.434109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:34:53.434109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:34:53.434109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:34:53.434109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:34:53.434109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:53.434109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:53.436750 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 285/590] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:53.437986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f016d61-3578-4da7-9b67-8ee52b23fa16 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:53.445338 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e8ed5-e759-4e36-9c05-fc4bae7ff8db None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:53.447694 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:53.482451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c437da-b341-4360-a8ee-7a92d13674ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:53.483125 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c437da-b341-4360-a8ee-7a92d13674ee None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:53.483413 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c437da-b341-4360-a8ee-7a92d13674ee None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:53.483695 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c437da-b341-4360-a8ee-7a92d13674ee None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:53.484942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c437da-b341-4360-a8ee-7a92d13674ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:53.484942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c437da-b341-4360-a8ee-7a92d13674ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:53.486132 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c437da-b341-4360-a8ee-7a92d13674ee None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:53.504939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 306/591] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:53.505923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c437da-b341-4360-a8ee-7a92d13674ee None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:53.570996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f016d61-3578-4da7-9b67-8ee52b23fa16 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:53.609067 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85cf1f6-6ee3-42f3-9d16-b6bbf6f1d095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:53.609866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85cf1f6-6ee3-42f3-9d16-b6bbf6f1d095 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:53.610150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85cf1f6-6ee3-42f3-9d16-b6bbf6f1d095 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:53.610437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85cf1f6-6ee3-42f3-9d16-b6bbf6f1d095 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:53.621170 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85cf1f6-6ee3-42f3-9d16-b6bbf6f1d095 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:53.622899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85cf1f6-6ee3-42f3-9d16-b6bbf6f1d095 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:53.640157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-f85cf1f6-6ee3-42f3-9d16-b6bbf6f1d095 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 27 15:34:53.640157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:53.640157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:53.640157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:53.640157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:53.640157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:53.640157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:53.640157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:53.640157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:53.640157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:53.640157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:53.640157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:53.640157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:34:53.640157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:34:53.640157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:34:53.640157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:34:53.641685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:53.641685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:53.641685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:34:53.641685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:53.641685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:34:53.641685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:53.641685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:53.641685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:53.641685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:53.641685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:53.641685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:53.641685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:53.641685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:53.641685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:53.641685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:53.641685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:53.641685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:53.641685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:53.642889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:53.642889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:53.642889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:34:53.642889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:34:53.642889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:34:53.642889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:34:53.642889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:34:53.642889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:34:53.642889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 27 15:34:53.642889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:53.642889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 286/592] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:53.642889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85cf1f6-6ee3-42f3-9d16-b6bbf6f1d095 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:53.650452 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c437da-b341-4360-a8ee-7a92d13674ee None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:53.695263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f9a47-e497-44c6-b332-af8bfcac63cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:53.696962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f9a47-e497-44c6-b332-af8bfcac63cd None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:53.697615 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f9a47-e497-44c6-b332-af8bfcac63cd None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:53.698368 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f9a47-e497-44c6-b332-af8bfcac63cd None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:53.699429 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f9a47-e497-44c6-b332-af8bfcac63cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:53.699904 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f9a47-e497-44c6-b332-af8bfcac63cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:53.701513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f9a47-e497-44c6-b332-af8bfcac63cd None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:53.707430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:53.721272 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 307/593] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:34:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:53.723019 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f9a47-e497-44c6-b332-af8bfcac63cd None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:53.758016 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:53.774659 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85cf1f6-6ee3-42f3-9d16-b6bbf6f1d095 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:53.826145 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39236496-d436-42a1-931a-649e257f987c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:53.827091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39236496-d436-42a1-931a-649e257f987c None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:53.827470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39236496-d436-42a1-931a-649e257f987c None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:53.827856 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39236496-d436-42a1-931a-649e257f987c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:53.839046 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39236496-d436-42a1-931a-649e257f987c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:53.842144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39236496-d436-42a1-931a-649e257f987c None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:53.855190 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f9a47-e497-44c6-b332-af8bfcac63cd None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:53.860639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-39236496-d436-42a1-931a-649e257f987c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 15:34:53.860639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:53.860639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:53.860639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:53.860639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:53.860639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:53.860639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:53.860639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:53.860639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:53.860639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:53.860639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:53.860639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:53.860639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:34:53.860639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:34:53.860639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:34:53.860639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:34:53.862234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:53.862234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:53.862234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:34:53.862234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:53.862234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:34:53.862234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:53.862234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:53.862234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:53.862234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:53.862234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:53.862234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:53.862234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:53.862234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:53.862234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:53.862234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:53.862234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:53.862234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:53.862234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:53.863666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:53.863666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:53.863666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:34:53.863666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:34:53.863666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:34:53.863666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:34:53.863666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:34:53.863666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:34:53.863666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 15:34:53.863666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:53.864453 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 287/594] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:34:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:53.865295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39236496-d436-42a1-931a-649e257f987c None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:53.910285 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047b324-edd1-4237-934f-0bec6179063f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:53.911291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047b324-edd1-4237-934f-0bec6179063f None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:53.911811 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047b324-edd1-4237-934f-0bec6179063f None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:53.912292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047b324-edd1-4237-934f-0bec6179063f None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:53.913427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047b324-edd1-4237-934f-0bec6179063f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:53.914240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047b324-edd1-4237-934f-0bec6179063f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:53.916124 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047b324-edd1-4237-934f-0bec6179063f None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:53.939689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 308/595] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Jun 27 15:34:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:53.941096 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047b324-edd1-4237-934f-0bec6179063f None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:53.987885 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39236496-d436-42a1-931a-649e257f987c None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:54.000269 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:54.035328 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd69ff-33c4-41d4-aca3-ba8a03d62013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:54.036305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd69ff-33c4-41d4-aca3-ba8a03d62013 None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:54.036589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd69ff-33c4-41d4-aca3-ba8a03d62013 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:54.036873 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd69ff-33c4-41d4-aca3-ba8a03d62013 None admin] PATH_INFO: `/v3/projects/7be83fb6f20341139147373837320148/users/51404afd35b94c47ba5c44f0deb73e30/roles/805851682adf4debbe99b1681c9fcdfc` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:54.054256 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd69ff-33c4-41d4-aca3-ba8a03d62013 None admin] RBAC: Authorizing `identity:create_grant(project_id=7be83fb6f20341139147373837320148, user_id=51404afd35b94c47ba5c44f0deb73e30, role_id=805851682adf4debbe99b1681c9fcdfc)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:54.056546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd69ff-33c4-41d4-aca3-ba8a03d62013 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:54.060562 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:54.076179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 288/596] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:34:53 2022] PUT /identity/v3/projects/7be83fb6f20341139147373837320148/users/51404afd35b94c47ba5c44f0deb73e30/roles/805851682adf4debbe99b1681c9fcdfc => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:54.077523 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd69ff-33c4-41d4-aca3-ba8a03d62013 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:54.090901 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047b324-edd1-4237-934f-0bec6179063f None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:54.141456 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d0bd1-9e7d-4b00-ac27-dc31143464c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:54.142613 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d0bd1-9e7d-4b00-ac27-dc31143464c1 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:54.143120 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d0bd1-9e7d-4b00-ac27-dc31143464c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:54.143644 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d0bd1-9e7d-4b00-ac27-dc31143464c1 None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:54.144729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d0bd1-9e7d-4b00-ac27-dc31143464c1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:54.145257 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d0bd1-9e7d-4b00-ac27-dc31143464c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:54.147087 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d0bd1-9e7d-4b00-ac27-dc31143464c1 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:54.174465 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 309/597] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Jun 27 15:34:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:54.180938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d0bd1-9e7d-4b00-ac27-dc31143464c1 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:54.276309 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd69ff-33c4-41d4-aca3-ba8a03d62013 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:54.350840 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75ca81-8170-4fdc-a3ed-631fd1f23b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:54.352039 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75ca81-8170-4fdc-a3ed-631fd1f23b23 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:54.353880 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75ca81-8170-4fdc-a3ed-631fd1f23b23 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:54.355501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75ca81-8170-4fdc-a3ed-631fd1f23b23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:54.370598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75ca81-8170-4fdc-a3ed-631fd1f23b23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:54.372532 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75ca81-8170-4fdc-a3ed-631fd1f23b23 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:54.379965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d0bd1-9e7d-4b00-ac27-dc31143464c1 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:54.394963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-fb75ca81-8170-4fdc-a3ed-631fd1f23b23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:54.394963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:54.394963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:54.394963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:54.394963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:54.394963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:54.394963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:54.394963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:54.394963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:54.394963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:54.394963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:54.394963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:54.394963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:54.394963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:54.394963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:54.395879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:54.395879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:54.395879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:54.395879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:54.395879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:54.395879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:54.395879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:54.395879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:54.395879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:54.395879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:54.395879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:54.395879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:54.395879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:54.395879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:54.395879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:54.395879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:54.395879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:54.396758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:54.396758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:54.396758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:54.396758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:54.399053 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 289/598] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:54.405806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75ca81-8170-4fdc-a3ed-631fd1f23b23 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:54.421455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ce835-b173-412d-a7d6-fd98b9c21e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:54.421770 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ce835-b173-412d-a7d6-fd98b9c21e49 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:54.421861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ce835-b173-412d-a7d6-fd98b9c21e49 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:54.422228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ce835-b173-412d-a7d6-fd98b9c21e49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:54.430052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:54.432809 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:54.435343 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ce835-b173-412d-a7d6-fd98b9c21e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:54.436900 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ce835-b173-412d-a7d6-fd98b9c21e49 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:54.481279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-c70ce835-b173-412d-a7d6-fd98b9c21e49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:54.481279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:54.481279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:54.481279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:54.481279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:54.481279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:54.481279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:54.481279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:54.481279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:54.481279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:54.481279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:54.481279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:54.481279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:54.481279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:54.481279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:54.482474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:54.482474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:54.482474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:54.482474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:54.482474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:54.482474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:54.482474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:54.482474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:54.482474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:54.482474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:54.482474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:54.482474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:54.482474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:54.482474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:54.482474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:54.482474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:54.482474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:54.483372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:54.483372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:54.483372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:54.483372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:54.483372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 310/599] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:54.488898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ce835-b173-412d-a7d6-fd98b9c21e49 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:54.614694 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75ca81-8170-4fdc-a3ed-631fd1f23b23 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:54.680968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ce835-b173-412d-a7d6-fd98b9c21e49 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:54.681850 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd3eaf-cdef-4aa3-8dee-834269c9647f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:54.682534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd3eaf-cdef-4aa3-8dee-834269c9647f None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:54.682816 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd3eaf-cdef-4aa3-8dee-834269c9647f None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:54.683092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd3eaf-cdef-4aa3-8dee-834269c9647f None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:54.686719 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd3eaf-cdef-4aa3-8dee-834269c9647f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:54.686719 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd3eaf-cdef-4aa3-8dee-834269c9647f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:54.686719 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd3eaf-cdef-4aa3-8dee-834269c9647f None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:54.698478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 290/600] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:54.704948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd3eaf-cdef-4aa3-8dee-834269c9647f None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:54.758363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818704a-0d32-478b-a3c4-19dcbdbcd774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:54.759583 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818704a-0d32-478b-a3c4-19dcbdbcd774 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:54.759866 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818704a-0d32-478b-a3c4-19dcbdbcd774 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:54.760136 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818704a-0d32-478b-a3c4-19dcbdbcd774 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:54.760874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818704a-0d32-478b-a3c4-19dcbdbcd774 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:54.761178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818704a-0d32-478b-a3c4-19dcbdbcd774 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:54.762585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818704a-0d32-478b-a3c4-19dcbdbcd774 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:54.784020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 311/601] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:54.795828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818704a-0d32-478b-a3c4-19dcbdbcd774 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:54.800943 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:54.838029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd3eaf-cdef-4aa3-8dee-834269c9647f None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:54.873521 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f21726-4461-493d-90f3-82506ddb928e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:54.874371 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f21726-4461-493d-90f3-82506ddb928e None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:54.874786 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f21726-4461-493d-90f3-82506ddb928e None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:54.875118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f21726-4461-493d-90f3-82506ddb928e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:54.882669 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:54.883676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f21726-4461-493d-90f3-82506ddb928e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:54.886195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f21726-4461-493d-90f3-82506ddb928e None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:54.903414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-b5f21726-4461-493d-90f3-82506ddb928e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:54.903414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:54.903414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:54.903414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:54.903414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:54.903414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:54.903414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:54.903414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:54.903414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:54.903414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:54.903414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:54.903414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:54.903414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:34:54.903414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:34:54.903414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:34:54.904525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:34:54.904525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:54.904525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:54.904525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:54.904525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:54.904525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:54.904525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:54.904525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:54.904525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:54.904525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:54.904525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:54.904525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:54.904525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:54.904525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:54.904525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:54.904525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:54.904525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:54.905621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:34:54.905621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:34:54.905621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:34:54.905621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:34:54.905621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:34:54.905621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:34:54.905621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:54.905621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:54.909666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 291/602] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:34:54.911928 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f21726-4461-493d-90f3-82506ddb928e None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:55.017982 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818704a-0d32-478b-a3c4-19dcbdbcd774 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:55.024801 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f21726-4461-493d-90f3-82506ddb928e None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:55.062159 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5131c18-5214-448a-a576-2054bd5b09c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:55.062945 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5131c18-5214-448a-a576-2054bd5b09c0 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.063231 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5131c18-5214-448a-a576-2054bd5b09c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.063505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5131c18-5214-448a-a576-2054bd5b09c0 None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.064363 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5131c18-5214-448a-a576-2054bd5b09c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:55.064672 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5131c18-5214-448a-a576-2054bd5b09c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:55.066199 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5131c18-5214-448a-a576-2054bd5b09c0 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:55.074691 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:55.080744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 292/603] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:34:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:55.086174 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5131c18-5214-448a-a576-2054bd5b09c0 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:55.112164 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0729d92-b5dc-4788-992f-7d15c1d915ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:55.113009 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0729d92-b5dc-4788-992f-7d15c1d915ac None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.113293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0729d92-b5dc-4788-992f-7d15c1d915ac None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.113565 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0729d92-b5dc-4788-992f-7d15c1d915ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.128492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0729d92-b5dc-4788-992f-7d15c1d915ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:55.130331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0729d92-b5dc-4788-992f-7d15c1d915ac None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:55.149956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-c0729d92-b5dc-4788-992f-7d15c1d915ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:55.149956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:55.149956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:55.149956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:55.149956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:55.149956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:55.149956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:55.149956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:55.149956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:55.149956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:55.149956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:55.149956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:55.149956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:34:55.149956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:34:55.149956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:34:55.150886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:34:55.150886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:55.150886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:55.150886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:55.150886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:55.150886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:55.150886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:55.150886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:55.150886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:55.150886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:55.150886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:55.150886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:55.150886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:55.150886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:55.150886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:55.150886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:55.150886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:55.151747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:34:55.151747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:34:55.151747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:34:55.151747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:34:55.151747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:34:55.151747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:34:55.151747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:55.151747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:55.151747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 312/604] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:55.152767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9d5cc-a68e-4cac-a4f3-40dfa5f29abb None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.153052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9d5cc-a68e-4cac-a4f3-40dfa5f29abb None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.153325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9d5cc-a68e-4cac-a4f3-40dfa5f29abb None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.158806 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 313/605] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:55.161797 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9d5cc-a68e-4cac-a4f3-40dfa5f29abb None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:55.228373 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5131c18-5214-448a-a576-2054bd5b09c0 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:55.266561 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5f0b9-e69d-4fed-832d-95723637468e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:55.267638 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5f0b9-e69d-4fed-832d-95723637468e None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.268042 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5f0b9-e69d-4fed-832d-95723637468e None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.268325 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5f0b9-e69d-4fed-832d-95723637468e None admin] PATH_INFO: `/v3/projects/7be83fb6f20341139147373837320148/users/493fcde4a45944efbc0b52c44e61b411/roles/805851682adf4debbe99b1681c9fcdfc` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.275298 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9d5cc-a68e-4cac-a4f3-40dfa5f29abb None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:55.286185 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5f0b9-e69d-4fed-832d-95723637468e None admin] RBAC: Authorizing `identity:create_grant(project_id=7be83fb6f20341139147373837320148, user_id=493fcde4a45944efbc0b52c44e61b411, role_id=805851682adf4debbe99b1681c9fcdfc)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:55.286977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:55.289436 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5f0b9-e69d-4fed-832d-95723637468e None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:55.313329 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 293/606] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:34:55 2022] PUT /identity/v3/projects/7be83fb6f20341139147373837320148/users/493fcde4a45944efbc0b52c44e61b411/roles/805851682adf4debbe99b1681c9fcdfc => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:55.315101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd935e-2337-4ea4-a933-64b1493d4cf3 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.315540 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd935e-2337-4ea4-a933-64b1493d4cf3 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.316044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd935e-2337-4ea4-a933-64b1493d4cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.318156 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef88346-43d0-486c-aab9-0ae831828423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:55.319135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef88346-43d0-486c-aab9-0ae831828423 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.319551 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef88346-43d0-486c-aab9-0ae831828423 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.322220 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef88346-43d0-486c-aab9-0ae831828423 None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.325958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef88346-43d0-486c-aab9-0ae831828423 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:55.326511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef88346-43d0-486c-aab9-0ae831828423 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:55.329220 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef88346-43d0-486c-aab9-0ae831828423 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:55.357673 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-f4cd935e-2337-4ea4-a933-64b1493d4cf3 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:55.357673 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f4cd935e-2337-4ea4-a933-64b1493d4cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:55.358322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 314/607] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:34:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:55.360403 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef88346-43d0-486c-aab9-0ae831828423 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:55.452427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:55.501532 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 294/608] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:55.503561 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa39c7-75f3-4976-bcee-12a4e03aaa0d None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.504116 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa39c7-75f3-4976-bcee-12a4e03aaa0d None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.505620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa39c7-75f3-4976-bcee-12a4e03aaa0d None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.507931 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 295/609] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:55.513482 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154c922-a21b-4e09-9ce1-39087f9624d5 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.513911 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154c922-a21b-4e09-9ce1-39087f9624d5 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.515002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154c922-a21b-4e09-9ce1-39087f9624d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.519415 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef88346-43d0-486c-aab9-0ae831828423 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:55.544238 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-8154c922-a21b-4e09-9ce1-39087f9624d5 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:55.545751 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-8154c922-a21b-4e09-9ce1-39087f9624d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:55.560805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c403537-c756-4512-8f00-ebba570d2de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:55.561979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c403537-c756-4512-8f00-ebba570d2de4 None admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.562416 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c403537-c756-4512-8f00-ebba570d2de4 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.562842 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c403537-c756-4512-8f00-ebba570d2de4 None admin] PATH_INFO: `/v3/projects/7be83fb6f20341139147373837320148/users/42dee639e2834d99b7468f911c37cf8f/roles/805851682adf4debbe99b1681c9fcdfc` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.580456 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c403537-c756-4512-8f00-ebba570d2de4 None admin] RBAC: Authorizing `identity:create_grant(project_id=7be83fb6f20341139147373837320148, user_id=42dee639e2834d99b7468f911c37cf8f, role_id=805851682adf4debbe99b1681c9fcdfc)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:55.582576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c403537-c756-4512-8f00-ebba570d2de4 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:55.598444 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 315/610] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:34:55 2022] PUT /identity/v3/projects/7be83fb6f20341139147373837320148/users/42dee639e2834d99b7468f911c37cf8f/roles/805851682adf4debbe99b1681c9fcdfc => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:55.599660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c403537-c756-4512-8f00-ebba570d2de4 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:55.624799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:55.689850 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 296/611] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:34:55.750663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c403537-c756-4512-8f00-ebba570d2de4 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:55.755432 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154c922-a21b-4e09-9ce1-39087f9624d5 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:55.808641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04469c6-69b9-4b76-8ecf-4b26e5ed24e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:55.809539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04469c6-69b9-4b76-8ecf-4b26e5ed24e3 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.810746 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04469c6-69b9-4b76-8ecf-4b26e5ed24e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.811037 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04469c6-69b9-4b76-8ecf-4b26e5ed24e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.824151 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04469c6-69b9-4b76-8ecf-4b26e5ed24e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:55.834253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-c04469c6-69b9-4b76-8ecf-4b26e5ed24e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:55.834253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:55.834253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:55.834253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:55.834253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:55.834253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:55.834253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:55.834253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:55.834253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:55.834253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:55.834253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:55.834253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:55.834253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:55.834253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:55.834253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:55.834253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:55.835393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:55.835393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:55.835393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:55.835393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:55.835393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:55.835393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:55.835393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:55.835393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:55.835393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:55.835393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:55.835393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:55.835393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:55.835393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:55.835393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:55.835393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:55.835393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:55.836399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:55.836399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:55.836399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:55.836399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:55.836399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:55.836399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:55.836399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:55.836399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:55.836399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:55.836399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:55.836399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:55.836399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:55.836399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 316/612] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:34:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:55.841811 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04469c6-69b9-4b76-8ecf-4b26e5ed24e3 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:55.894471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154c922-a21b-4e09-9ce1-39087f9624d5 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:55.896237 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:55.943745 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b03707-caff-4903-8c18-b78a4555ba3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:55.944626 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b03707-caff-4903-8c18-b78a4555ba3e None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:55.945291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b03707-caff-4903-8c18-b78a4555ba3e None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:55.945291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b03707-caff-4903-8c18-b78a4555ba3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:55.952306 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:55.976454 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b03707-caff-4903-8c18-b78a4555ba3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:55.976820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04469c6-69b9-4b76-8ecf-4b26e5ed24e3 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:55.986546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-58b03707-caff-4903-8c18-b78a4555ba3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:55.986546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:55.986546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:55.986546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:55.986546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:55.986546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:55.986546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:55.986546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:55.986546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:55.986546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:55.986546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:55.986546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:55.986546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:55.986546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:55.986546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:55.986546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:55.987645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:55.987645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:55.987645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:55.987645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:55.987645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:55.987645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:55.987645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:55.987645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:55.987645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:55.987645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:55.987645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:55.987645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:55.987645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:55.987645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:55.987645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:55.987645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:55.988483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:55.988483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:55.988483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:55.988483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:55.988483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:55.988483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:55.988483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:55.988483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:55.988483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:55.988483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:55.988483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:55.988483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:55.993925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 297/613] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:34:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:34:55.997869 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b03707-caff-4903-8c18-b78a4555ba3e None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.050174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a1b7f-1bff-4926-bf4f-faa6c205eb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:56.051130 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a1b7f-1bff-4926-bf4f-faa6c205eb34 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.051556 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a1b7f-1bff-4926-bf4f-faa6c205eb34 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.052137 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a1b7f-1bff-4926-bf4f-faa6c205eb34 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.053163 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a1b7f-1bff-4926-bf4f-faa6c205eb34 None admin] RBAC: Adding query filter params (name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:56.053688 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a1b7f-1bff-4926-bf4f-faa6c205eb34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:56.055203 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725a1b7f-1bff-4926-bf4f-faa6c205eb34 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:56.066599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 317/614] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:34:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:56.073542 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a1b7f-1bff-4926-bf4f-faa6c205eb34 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.190490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a1b7f-1bff-4926-bf4f-faa6c205eb34 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:56.228382 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b03707-caff-4903-8c18-b78a4555ba3e None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:56.230036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c82b4-6f27-453d-abae-7d437ff956bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:56.231038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c82b4-6f27-453d-abae-7d437ff956bc None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.231462 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c82b4-6f27-453d-abae-7d437ff956bc None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.231908 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c82b4-6f27-453d-abae-7d437ff956bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.244484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c82b4-6f27-453d-abae-7d437ff956bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:56.246204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c82b4-6f27-453d-abae-7d437ff956bc None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:56.248396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:56.267322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-5e6c82b4-6f27-453d-abae-7d437ff956bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:56.267322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:56.267322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:56.267322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:56.267322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:56.267322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:56.267322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:56.267322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:56.267322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:56.267322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:56.267322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:56.267322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:56.267322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:56.267322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:56.267322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:56.268300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:56.268300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:56.268300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:56.268300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:56.268300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:56.268300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:56.268300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:56.268300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:56.268300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:56.268300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:56.268300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:56.268300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:56.268300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:56.268300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:56.268300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:56.268300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:56.268300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:56.269145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:56.269145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:56.269145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:56.269145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:56.271449 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 318/615] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:56.277386 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c82b4-6f27-453d-abae-7d437ff956bc None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.279168 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6e63b-9dfa-4893-b603-058acfa87ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:56.279851 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6e63b-9dfa-4893-b603-058acfa87ab2 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.280139 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6e63b-9dfa-4893-b603-058acfa87ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.280417 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6e63b-9dfa-4893-b603-058acfa87ab2 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.281176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6e63b-9dfa-4893-b603-058acfa87ab2 None admin] RBAC: Adding query filter params (name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:56.281483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6e63b-9dfa-4893-b603-058acfa87ab2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:56.283662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6e63b-9dfa-4893-b603-058acfa87ab2 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:56.299990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 298/616] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:34:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:56.305507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6e63b-9dfa-4893-b603-058acfa87ab2 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.362735 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:56.396000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c82b4-6f27-453d-abae-7d437ff956bc None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:56.436657 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a3bcd-d030-498c-9b32-768f7a0b5c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:56.437732 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a3bcd-d030-498c-9b32-768f7a0b5c2f None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.438242 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a3bcd-d030-498c-9b32-768f7a0b5c2f None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.438714 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a3bcd-d030-498c-9b32-768f7a0b5c2f None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.439640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a3bcd-d030-498c-9b32-768f7a0b5c2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:56.445659 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a3bcd-d030-498c-9b32-768f7a0b5c2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:56.447600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a3bcd-d030-498c-9b32-768f7a0b5c2f None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:56.472112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 319/617] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:34:56.477492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6e63b-9dfa-4893-b603-058acfa87ab2 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:56.478624 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a3bcd-d030-498c-9b32-768f7a0b5c2f None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.516725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c1a62-b99d-43b7-b7f6-6e6619704cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:56.517754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c1a62-b99d-43b7-b7f6-6e6619704cc6 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.518037 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c1a62-b99d-43b7-b7f6-6e6619704cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.518316 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c1a62-b99d-43b7-b7f6-6e6619704cc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.531007 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c1a62-b99d-43b7-b7f6-6e6619704cc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:56.532486 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c1a62-b99d-43b7-b7f6-6e6619704cc6 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:56.554087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-471c1a62-b99d-43b7-b7f6-6e6619704cc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:56.554087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:56.554087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:56.554087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:56.554087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:56.554087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:56.554087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:56.554087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:56.554087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:56.554087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:56.554087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:56.554087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:56.554087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:56.554087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:56.554087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:56.555013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:56.555013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:56.555013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:56.555013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:56.555013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:56.555013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:56.555013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:56.555013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:56.555013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:56.555013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:56.555013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:56.555013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:56.555013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:56.555013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:56.555013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:56.555013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:56.555013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:56.556473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:56.556473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:56.556473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:56.556473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:56.556473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 299/618] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:34:56.563774 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c1a62-b99d-43b7-b7f6-6e6619704cc6 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.586625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:56.644709 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a3bcd-d030-498c-9b32-768f7a0b5c2f None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:56.665915 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:56.683362 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce697c7-81b6-42e1-b233-6a547742663d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:56.685240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce697c7-81b6-42e1-b233-6a547742663d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.685684 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce697c7-81b6-42e1-b233-6a547742663d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.686175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce697c7-81b6-42e1-b233-6a547742663d None admin] PATH_INFO: `/v3/users/swift` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.692056 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c1a62-b99d-43b7-b7f6-6e6619704cc6 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:56.694295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce697c7-81b6-42e1-b233-6a547742663d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:56.695944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce697c7-81b6-42e1-b233-6a547742663d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:56.712585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-2ce697c7-81b6-42e1-b233-6a547742663d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:34:56.712585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:56.712585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:56.712585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:56.712585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:56.712585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:56.712585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:56.712585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:56.712585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:56.712585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:56.712585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:56.712585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:56.712585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:34:56.712585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:34:56.712585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:34:56.712585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:34:56.713778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:56.713778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:56.713778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:34:56.713778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:56.713778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:34:56.713778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:56.713778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:56.713778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:56.713778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:56.713778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:56.713778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:56.713778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:56.713778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:56.713778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:56.713778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:56.713778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:56.713778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:56.713778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:56.715075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:56.715075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:56.715075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:34:56.715075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:34:56.715075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:34:56.715075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:34:56.715075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:34:56.715075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:34:56.715075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:34:56.715075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:56.717539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 320/619] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:34:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:34:56.719016 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0bb27-8db0-44fd-853f-bcfeb4cfd615 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.719454 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0bb27-8db0-44fd-853f-bcfeb4cfd615 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.719869 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0bb27-8db0-44fd-853f-bcfeb4cfd615 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.725508 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 321/620] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:56.726509 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0bb27-8db0-44fd-853f-bcfeb4cfd615 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.734469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a18183-f864-4e9f-9cfc-949938a9ec8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:56.735230 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a18183-f864-4e9f-9cfc-949938a9ec8a None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.735627 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a18183-f864-4e9f-9cfc-949938a9ec8a None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.735972 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a18183-f864-4e9f-9cfc-949938a9ec8a None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.736860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a18183-f864-4e9f-9cfc-949938a9ec8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:56.737435 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a18183-f864-4e9f-9cfc-949938a9ec8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:56.738788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a18183-f864-4e9f-9cfc-949938a9ec8a None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:56.753047 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 300/621] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:56.754538 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d2f53-c6b0-43ba-b477-59139691868a None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.754996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d2f53-c6b0-43ba-b477-59139691868a None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.755348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d2f53-c6b0-43ba-b477-59139691868a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.774101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-790d2f53-c6b0-43ba-b477-59139691868a None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:56.776198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-790d2f53-c6b0-43ba-b477-59139691868a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:56.839978 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0bb27-8db0-44fd-853f-bcfeb4cfd615 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:56.857658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 301/622] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:56.857658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d2f53-c6b0-43ba-b477-59139691868a None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.869665 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:56.900219 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd5346-9526-4ae1-8148-2b5d8d8d81f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:56.900894 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd5346-9526-4ae1-8148-2b5d8d8d81f0 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:56.901242 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd5346-9526-4ae1-8148-2b5d8d8d81f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:56.901650 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd5346-9526-4ae1-8148-2b5d8d8d81f0 None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:56.902882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd5346-9526-4ae1-8148-2b5d8d8d81f0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:56.903771 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd5346-9526-4ae1-8148-2b5d8d8d81f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:56.905567 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd5346-9526-4ae1-8148-2b5d8d8d81f0 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:56.930439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 322/623] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Jun 27 15:34:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:56.931578 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd5346-9526-4ae1-8148-2b5d8d8d81f0 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:56.945891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:57.001982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d2f53-c6b0-43ba-b477-59139691868a None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:57.048259 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfe8a9-dc3c-4907-853d-de88115a1fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:57.048837 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd5346-9526-4ae1-8148-2b5d8d8d81f0 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:57.049270 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfe8a9-dc3c-4907-853d-de88115a1fc6 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:57.049535 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfe8a9-dc3c-4907-853d-de88115a1fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:57.049915 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfe8a9-dc3c-4907-853d-de88115a1fc6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:57.059392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dfe8a9-dc3c-4907-853d-de88115a1fc6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:57.061335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dfe8a9-dc3c-4907-853d-de88115a1fc6 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:57.081684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-e2dfe8a9-dc3c-4907-853d-de88115a1fc6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:34:57.081684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:57.081684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:57.081684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:57.081684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:57.081684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:57.081684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:57.081684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:57.081684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:57.081684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:57.081684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:57.081684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:57.081684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:34:57.081684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:34:57.081684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:34:57.081684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:34:57.082680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:57.082680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:57.082680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:34:57.082680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:57.082680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:34:57.082680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:57.082680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:57.082680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:57.082680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:57.082680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:57.082680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:57.082680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:57.082680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:57.082680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:57.082680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:57.082680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:57.082680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:57.082680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:57.083605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:57.083605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:57.083605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:34:57.083605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:34:57.083605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:34:57.083605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:34:57.083605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:34:57.083605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:34:57.083605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:34:57.083605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:57.086333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 302/624] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Jun 27 15:34:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:57.088020 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfe8a9-dc3c-4907-853d-de88115a1fc6 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:57.097499 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c8639-3688-441d-8b37-918e085cd157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:57.098699 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c8639-3688-441d-8b37-918e085cd157 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:57.099180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c8639-3688-441d-8b37-918e085cd157 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:57.099649 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c8639-3688-441d-8b37-918e085cd157 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:57.111463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c8639-3688-441d-8b37-918e085cd157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:57.120152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-438c8639-3688-441d-8b37-918e085cd157 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:57.120152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:57.120152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:57.120152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:57.120152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:57.120152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:57.120152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:57.120152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:57.120152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:57.120152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:57.120152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:57.120152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:57.120152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:57.120152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:57.120152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:57.120152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:57.121085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:57.121085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:57.121085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:57.121085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:57.121085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:57.121085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:57.121085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:57.121085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:57.121085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:57.121085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:57.121085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:57.121085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:57.121085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:57.121085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:57.121085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:57.121085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:57.121944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:57.121944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:57.121944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:57.121944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:57.121944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:57.121944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:57.121944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:57.121944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:57.121944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:57.121944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:57.121944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:57.121944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:57.124623 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 323/625] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:34:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:57.125751 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c8639-3688-441d-8b37-918e085cd157 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:57.166129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:57.200820 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfe8a9-dc3c-4907-853d-de88115a1fc6 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:57.230974 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:57.241546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c8639-3688-441d-8b37-918e085cd157 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:57.247218 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b562a95-7f69-4401-9611-03ff30308acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:57.248287 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b562a95-7f69-4401-9611-03ff30308acf None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:57.248770 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b562a95-7f69-4401-9611-03ff30308acf None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:57.249285 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b562a95-7f69-4401-9611-03ff30308acf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:57.264562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b562a95-7f69-4401-9611-03ff30308acf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:57.266874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b562a95-7f69-4401-9611-03ff30308acf None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:57.279654 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40857595-e09a-4dac-b0c6-e40a55d7c9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:57.280327 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40857595-e09a-4dac-b0c6-e40a55d7c9ce None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:57.280611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40857595-e09a-4dac-b0c6-e40a55d7c9ce None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:57.280889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40857595-e09a-4dac-b0c6-e40a55d7c9ce None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:57.281777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40857595-e09a-4dac-b0c6-e40a55d7c9ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:57.282141 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40857595-e09a-4dac-b0c6-e40a55d7c9ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:57.283626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40857595-e09a-4dac-b0c6-e40a55d7c9ce None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:57.292605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-5b562a95-7f69-4401-9611-03ff30308acf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:57.292605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:57.292605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:57.292605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:57.292605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:57.292605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:57.292605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:57.292605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:57.292605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:57.292605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:57.292605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:57.292605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:57.292605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:57.292605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:57.292605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:57.293533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:57.293533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:57.293533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:57.293533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:57.293533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:57.293533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:57.293533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:57.293533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:57.293533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:57.293533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:57.293533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:57.293533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:57.293533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:57.293533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:57.293533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:57.293533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:57.293533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:57.294441 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:57.294441 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:57.294441 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:57.294441 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:57.296888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 303/626] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:57.297015 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b562a95-7f69-4401-9611-03ff30308acf None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:57.309253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 324/627] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Jun 27 15:34:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:57.314077 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40857595-e09a-4dac-b0c6-e40a55d7c9ce None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:57.426150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:57.436967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40857595-e09a-4dac-b0c6-e40a55d7c9ce None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:57.443051 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b562a95-7f69-4401-9611-03ff30308acf None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:57.473152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d173aec-cae5-444d-8496-5e352f3ad454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:57.473898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d173aec-cae5-444d-8496-5e352f3ad454 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:57.474172 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d173aec-cae5-444d-8496-5e352f3ad454 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:57.474498 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d173aec-cae5-444d-8496-5e352f3ad454 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:57.475171 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d173aec-cae5-444d-8496-5e352f3ad454 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:57.475483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d173aec-cae5-444d-8496-5e352f3ad454 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:57.476845 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d173aec-cae5-444d-8496-5e352f3ad454 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:57.490000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 325/628] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:57.491017 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d173aec-cae5-444d-8496-5e352f3ad454 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:57.491659 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9a065-fc86-4ad0-bcbf-64f7d8bf41ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:57.492545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9a065-fc86-4ad0-bcbf-64f7d8bf41ee None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:57.493043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9a065-fc86-4ad0-bcbf-64f7d8bf41ee None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:57.494119 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9a065-fc86-4ad0-bcbf-64f7d8bf41ee None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:57.496017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9a065-fc86-4ad0-bcbf-64f7d8bf41ee None admin] RBAC: Adding query filter params (name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:57.496550 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9a065-fc86-4ad0-bcbf-64f7d8bf41ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:57.498168 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9a065-fc86-4ad0-bcbf-64f7d8bf41ee None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:57.510130 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 304/629] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:34:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:57.514903 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9a065-fc86-4ad0-bcbf-64f7d8bf41ee None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:57.561718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:57.624211 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9a065-fc86-4ad0-bcbf-64f7d8bf41ee None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:57.624528 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d173aec-cae5-444d-8496-5e352f3ad454 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:57.662881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c9a44-47f3-4b42-a897-aaa86a7b0b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:57.663949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c9a44-47f3-4b42-a897-aaa86a7b0b78 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:57.664251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c9a44-47f3-4b42-a897-aaa86a7b0b78 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:57.664512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c9a44-47f3-4b42-a897-aaa86a7b0b78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:57.672552 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c9a44-47f3-4b42-a897-aaa86a7b0b78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:57.673112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b0e27-aa70-4c96-ab8c-26d28ce633ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:57.674018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b0e27-aa70-4c96-ab8c-26d28ce633ac None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:57.674126 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c9a44-47f3-4b42-a897-aaa86a7b0b78 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:57.674599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b0e27-aa70-4c96-ab8c-26d28ce633ac None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:57.674988 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b0e27-aa70-4c96-ab8c-26d28ce633ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:57.691377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-de2c9a44-47f3-4b42-a897-aaa86a7b0b78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:57.691377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:57.691377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:57.691377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:57.691377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:57.691377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:57.691377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:57.691377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:57.691377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:57.691377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:57.691377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:57.691377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:57.691377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:34:57.691377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:34:57.691377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:34:57.692283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:34:57.692283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:57.692283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:57.692283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:57.692283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:57.692283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:57.692283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:57.692283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:57.692283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:57.692283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:57.692283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:57.692283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:57.692283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:57.692283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:57.692283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:57.692283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:57.692283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:57.693176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:34:57.693176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:34:57.693176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:34:57.693176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:34:57.693176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:34:57.693176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:34:57.693176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:57.693176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:57.697314 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 305/630] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:57.698063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c9a44-47f3-4b42-a897-aaa86a7b0b78 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:57.700215 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:57.757716 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b0e27-aa70-4c96-ab8c-26d28ce633ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:57.760051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b0e27-aa70-4c96-ab8c-26d28ce633ac None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:57.804516 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:57.807886 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c9a44-47f3-4b42-a897-aaa86a7b0b78 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:57.845527 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4801f2-7883-4327-a037-c9c5e8e4945b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:57.846468 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4801f2-7883-4327-a037-c9c5e8e4945b None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:57.846756 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4801f2-7883-4327-a037-c9c5e8e4945b None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:57.847031 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4801f2-7883-4327-a037-c9c5e8e4945b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:57.858898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4801f2-7883-4327-a037-c9c5e8e4945b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:57.860365 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4801f2-7883-4327-a037-c9c5e8e4945b None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:57.881707 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 326/631] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:57.881881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-ee4801f2-7883-4327-a037-c9c5e8e4945b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:57.881881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:57.881881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:57.881881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:57.881881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:57.881881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:57.881881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:57.881881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:57.881881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:57.881881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:57.881881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:57.881881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:57.881881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:57.881881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:57.881881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:57.882728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:57.882728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:57.882728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:57.882728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:57.882728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:57.882728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:57.882728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:57.882728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:57.882728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:57.882728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:57.882728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:57.882728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:57.882728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:57.882728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:57.882728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:57.882728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:57.882728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:57.883624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:57.883624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:57.883624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:57.883624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:57.884411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a485a-91a9-49f4-94d7-e7f9993e9981 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:57.884411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a485a-91a9-49f4-94d7-e7f9993e9981 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:57.884411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a485a-91a9-49f4-94d7-e7f9993e9981 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:57.885587 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 306/632] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:57.886279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 327/633] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:57.886577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4801f2-7883-4327-a037-c9c5e8e4945b None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:57.887087 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a485a-91a9-49f4-94d7-e7f9993e9981 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:57.943093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:57.997203 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4801f2-7883-4327-a037-c9c5e8e4945b None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:57.999517 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a485a-91a9-49f4-94d7-e7f9993e9981 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:58.036068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910bae2-0bf1-464b-816f-cde2d56f2c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:58.036928 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910bae2-0bf1-464b-816f-cde2d56f2c2f None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:58.037208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910bae2-0bf1-464b-816f-cde2d56f2c2f None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:58.037491 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910bae2-0bf1-464b-816f-cde2d56f2c2f None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:58.038503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910bae2-0bf1-464b-816f-cde2d56f2c2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:58.039070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910bae2-0bf1-464b-816f-cde2d56f2c2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:58.039623 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916715c-99a8-4f33-bc8e-f51b36dd3650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:58.040823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910bae2-0bf1-464b-816f-cde2d56f2c2f None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:58.040950 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916715c-99a8-4f33-bc8e-f51b36dd3650 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:58.040950 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916715c-99a8-4f33-bc8e-f51b36dd3650 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:58.041103 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916715c-99a8-4f33-bc8e-f51b36dd3650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:58.058647 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 307/634] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:34:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:58.058839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916715c-99a8-4f33-bc8e-f51b36dd3650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:58.059278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910bae2-0bf1-464b-816f-cde2d56f2c2f None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:58.059392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916715c-99a8-4f33-bc8e-f51b36dd3650 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:58.076793 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:58.080104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-6916715c-99a8-4f33-bc8e-f51b36dd3650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:58.080104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:58.080104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:58.080104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:58.080104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:58.080104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:58.080104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:58.080104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:58.080104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:58.080104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:58.080104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:58.080104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:58.080104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:58.080104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:58.080104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:58.081194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:58.081194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:58.081194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:58.081194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:58.081194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:58.081194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:58.081194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:58.081194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:58.081194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:58.081194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:58.081194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:58.081194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:58.081194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:58.081194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:58.081194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:58.081194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:58.081194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:58.082123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:58.082123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:58.082123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:58.082123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:58.082123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 328/635] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:58.082508 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b280ff6-47e0-4def-920b-bd896bfbef79 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:58.082752 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b280ff6-47e0-4def-920b-bd896bfbef79 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:58.083030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b280ff6-47e0-4def-920b-bd896bfbef79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:58.099579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-8b280ff6-47e0-4def-920b-bd896bfbef79 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:58.100765 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8b280ff6-47e0-4def-920b-bd896bfbef79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:58.166716 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 329/636] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:58.167508 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b280ff6-47e0-4def-920b-bd896bfbef79 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:58.175377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910bae2-0bf1-464b-816f-cde2d56f2c2f None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:58.204926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:58.210196 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2ff9f-0867-4303-8586-bc882a77b7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:58.210696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2ff9f-0867-4303-8586-bc882a77b7c3 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:58.210984 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2ff9f-0867-4303-8586-bc882a77b7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:58.211263 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2ff9f-0867-4303-8586-bc882a77b7c3 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:58.212064 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2ff9f-0867-4303-8586-bc882a77b7c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:58.212352 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2ff9f-0867-4303-8586-bc882a77b7c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:58.213767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2ff9f-0867-4303-8586-bc882a77b7c3 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:58.225468 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 308/637] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:58.226293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2ff9f-0867-4303-8586-bc882a77b7c3 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:58.276732 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b280ff6-47e0-4def-920b-bd896bfbef79 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:58.295785 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:58.312502 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab34a4-258f-4e30-bebd-eb8cfcb1423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:58.313396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab34a4-258f-4e30-bebd-eb8cfcb1423a None admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:58.313723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab34a4-258f-4e30-bebd-eb8cfcb1423a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:58.314011 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab34a4-258f-4e30-bebd-eb8cfcb1423a None admin] PATH_INFO: `/v3/projects/7be83fb6f20341139147373837320148/users/8e83ea69ded040448fe75caa221ad377/roles/805851682adf4debbe99b1681c9fcdfc` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:58.325902 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2ff9f-0867-4303-8586-bc882a77b7c3 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:58.328020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab34a4-258f-4e30-bebd-eb8cfcb1423a None admin] RBAC: Authorizing `identity:create_grant(project_id=7be83fb6f20341139147373837320148, user_id=8e83ea69ded040448fe75caa221ad377, role_id=805851682adf4debbe99b1681c9fcdfc)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:58.329445 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab34a4-258f-4e30-bebd-eb8cfcb1423a None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:58.342155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 330/638] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:34:58 2022] PUT /identity/v3/projects/7be83fb6f20341139147373837320148/users/8e83ea69ded040448fe75caa221ad377/roles/805851682adf4debbe99b1681c9fcdfc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:34:58.343434 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15890c-7a8b-4b40-af20-300c6410c1a2 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:58.343729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15890c-7a8b-4b40-af20-300c6410c1a2 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:58.344012 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15890c-7a8b-4b40-af20-300c6410c1a2 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:58.345292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 331/639] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:34:58.346054 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15890c-7a8b-4b40-af20-300c6410c1a2 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:58.372524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136f627-9a7b-4ebd-9d6a-68dcf2ddac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:58.373224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136f627-9a7b-4ebd-9d6a-68dcf2ddac13 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:58.373510 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136f627-9a7b-4ebd-9d6a-68dcf2ddac13 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:58.374172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136f627-9a7b-4ebd-9d6a-68dcf2ddac13 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:58.374923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1136f627-9a7b-4ebd-9d6a-68dcf2ddac13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:58.375246 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1136f627-9a7b-4ebd-9d6a-68dcf2ddac13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:58.376679 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1136f627-9a7b-4ebd-9d6a-68dcf2ddac13 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:58.394543 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 309/640] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:58.394543 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136f627-9a7b-4ebd-9d6a-68dcf2ddac13 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:58.458765 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:58.481864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15890c-7a8b-4b40-af20-300c6410c1a2 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:58.507586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136f627-9a7b-4ebd-9d6a-68dcf2ddac13 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:58.525503 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c2f05-0f41-4ab9-88d9-54cf69d254d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:58.526341 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c2f05-0f41-4ab9-88d9-54cf69d254d0 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:58.526627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c2f05-0f41-4ab9-88d9-54cf69d254d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:58.526934 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c2f05-0f41-4ab9-88d9-54cf69d254d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:58.537614 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c2f05-0f41-4ab9-88d9-54cf69d254d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:58.542617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:58.546653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-761c2f05-0f41-4ab9-88d9-54cf69d254d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:58.546653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:58.546653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:58.546653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:58.546653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:58.546653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:58.546653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:58.546653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:58.546653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:58.546653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:58.546653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:58.546653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:58.546653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:58.546653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:58.546653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:58.546653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:58.547599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:58.547599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:58.547599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:58.547599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:58.547599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:58.547599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:58.547599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:58.547599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:58.547599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:58.547599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:58.547599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:58.547599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:58.547599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:58.547599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:58.547599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:58.547599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:58.548905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:58.548905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:58.548905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:58.548905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:58.548905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:58.548905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:58.548905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:58.548905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:58.548905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:58.548905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:58.548905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:58.548905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:58.548905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 332/641] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:34:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:58.549863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5464c-5dbc-4ea9-aaa8-dee30af0f40e None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:58.549863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5464c-5dbc-4ea9-aaa8-dee30af0f40e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:58.549863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5464c-5dbc-4ea9-aaa8-dee30af0f40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:58.553489 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ae3dd-684d-4749-831a-18438c624639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:58.554431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ae3dd-684d-4749-831a-18438c624639 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:58.554897 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ae3dd-684d-4749-831a-18438c624639 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:58.555013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ae3dd-684d-4749-831a-18438c624639 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:58.564454 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ae3dd-684d-4749-831a-18438c624639 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:58.565973 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ae3dd-684d-4749-831a-18438c624639 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:58.572895 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-faf5464c-5dbc-4ea9-aaa8-dee30af0f40e None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:34:58.575069 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-faf5464c-5dbc-4ea9-aaa8-dee30af0f40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:34:58.580748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-5b7ae3dd-684d-4749-831a-18438c624639 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 15:34:58.580748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:58.580748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:58.580748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:58.580748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:58.580748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:58.580748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:58.580748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:58.580748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:58.580748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:58.580748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:58.580748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:58.580748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:34:58.580748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:34:58.580748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:34:58.580748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:34:58.582598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:58.582598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:58.582598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:34:58.582598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:58.582598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:34:58.582598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:58.582598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:58.582598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:58.582598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:58.582598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:58.582598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:58.582598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:58.582598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:58.582598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:58.582598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:58.582598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:58.582598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:58.582598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:58.584597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:58.584597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:58.584597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:34:58.584597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:34:58.584597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:34:58.584597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:34:58.584597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:34:58.584597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:34:58.584597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 15:34:58.584597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:58.584597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 310/642] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:58.584597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ae3dd-684d-4749-831a-18438c624639 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:58.655385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 333/643] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:34:58.656365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5464c-5dbc-4ea9-aaa8-dee30af0f40e None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:58.711604 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ae3dd-684d-4749-831a-18438c624639 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:58.735953 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:58.748554 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fc6d7-0ec3-4f22-9e39-00b880335f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:58.749361 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fc6d7-0ec3-4f22-9e39-00b880335f11 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:58.749655 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fc6d7-0ec3-4f22-9e39-00b880335f11 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:58.749945 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fc6d7-0ec3-4f22-9e39-00b880335f11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:58.756870 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382fc6d7-0ec3-4f22-9e39-00b880335f11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:58.758345 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382fc6d7-0ec3-4f22-9e39-00b880335f11 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:58.769252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5464c-5dbc-4ea9-aaa8-dee30af0f40e None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:58.772625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-382fc6d7-0ec3-4f22-9e39-00b880335f11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:58.772625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:58.772625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:58.772625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:58.772625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:58.772625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:58.772625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:58.772625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:58.772625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:58.772625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:58.772625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:58.772625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:58.772625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:34:58.772625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:34:58.772625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:34:58.773503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:34:58.773503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:58.773503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:58.773503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:58.773503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:58.773503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:58.773503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:58.773503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:58.773503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:58.773503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:58.773503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:58.773503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:58.773503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:58.773503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:58.773503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:58.773503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:58.773503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:58.776054 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:34:58.776054 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:34:58.776054 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:34:58.776054 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:34:58.776054 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:34:58.776054 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:34:58.776054 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:34:58.776054 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:58.779106 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 311/644] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:34:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:58.779743 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fc6d7-0ec3-4f22-9e39-00b880335f11 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:58.811939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded25d26-683a-4ae8-ad1c-8abb6395a765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:58.812777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded25d26-683a-4ae8-ad1c-8abb6395a765 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:58.813379 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded25d26-683a-4ae8-ad1c-8abb6395a765 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:58.813689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded25d26-683a-4ae8-ad1c-8abb6395a765 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:58.814593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded25d26-683a-4ae8-ad1c-8abb6395a765 None admin] RBAC: Adding query filter params (name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:58.814907 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded25d26-683a-4ae8-ad1c-8abb6395a765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:58.816287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded25d26-683a-4ae8-ad1c-8abb6395a765 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:58.829404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 334/645] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:34:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:58.830241 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded25d26-683a-4ae8-ad1c-8abb6395a765 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:58.840399 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:58.895517 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fc6d7-0ec3-4f22-9e39-00b880335f11 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:58.934393 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13b494-71ed-4c7d-a9a0-c1370fe8d780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:58.935410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13b494-71ed-4c7d-a9a0-c1370fe8d780 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:58.935668 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded25d26-683a-4ae8-ad1c-8abb6395a765 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:58.936140 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13b494-71ed-4c7d-a9a0-c1370fe8d780 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:58.936625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13b494-71ed-4c7d-a9a0-c1370fe8d780 None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:58.937707 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13b494-71ed-4c7d-a9a0-c1370fe8d780 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:58.938233 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13b494-71ed-4c7d-a9a0-c1370fe8d780 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:58.939763 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13b494-71ed-4c7d-a9a0-c1370fe8d780 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:58.965742 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 312/646] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Jun 27 15:34:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:58.965742 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13b494-71ed-4c7d-a9a0-c1370fe8d780 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:58.970934 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbf42e-6547-4aa5-a2e5-2d50b055a890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:58.971733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbf42e-6547-4aa5-a2e5-2d50b055a890 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:58.972020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbf42e-6547-4aa5-a2e5-2d50b055a890 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:58.972297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbf42e-6547-4aa5-a2e5-2d50b055a890 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:58.974408 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:58.985758 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbf42e-6547-4aa5-a2e5-2d50b055a890 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:34:58.993667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-91bbf42e-6547-4aa5-a2e5-2d50b055a890 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:58.993667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:58.993667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:58.993667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:58.993667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:58.993667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:58.993667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:58.993667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:58.993667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:58.993667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:58.993667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:58.993667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:58.993667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:34:58.993667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:34:58.993667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:34:58.993667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:34:58.994580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:34:58.994580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:34:58.994580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:34:58.994580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:34:58.994580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:58.994580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:58.994580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:58.994580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:58.994580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:58.994580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:58.994580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:58.994580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:58.994580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:58.994580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:58.994580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:58.994580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:58.995385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:58.995385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:58.995385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:58.995385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:58.995385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:34:58.995385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:34:58.995385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:34:58.995385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:34:58.995385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:34:58.995385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:34:58.995385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:34:58.995385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:58.995385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 335/647] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:34:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:58.996096 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbf42e-6547-4aa5-a2e5-2d50b055a890 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:59.082470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13b494-71ed-4c7d-a9a0-c1370fe8d780 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:59.095840 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:59.115302 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbf42e-6547-4aa5-a2e5-2d50b055a890 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:59.124937 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a45dc-bf4b-426b-a6ae-920b89e51c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:59.125596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a45dc-bf4b-426b-a6ae-920b89e51c96 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:59.125904 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a45dc-bf4b-426b-a6ae-920b89e51c96 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:59.126177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a45dc-bf4b-426b-a6ae-920b89e51c96 None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:59.126967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a45dc-bf4b-426b-a6ae-920b89e51c96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:59.127269 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a45dc-bf4b-426b-a6ae-920b89e51c96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:59.128697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a45dc-bf4b-426b-a6ae-920b89e51c96 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:59.142197 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 313/648] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:34:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:59.147418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a45dc-bf4b-426b-a6ae-920b89e51c96 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:59.159113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0013eab-4893-441d-958c-fee45f6a7cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:59.159915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0013eab-4893-441d-958c-fee45f6a7cd8 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:59.160505 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0013eab-4893-441d-958c-fee45f6a7cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:59.160882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0013eab-4893-441d-958c-fee45f6a7cd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:59.172128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0013eab-4893-441d-958c-fee45f6a7cd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:59.173560 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0013eab-4893-441d-958c-fee45f6a7cd8 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:59.192483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-d0013eab-4893-441d-958c-fee45f6a7cd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:59.192483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:59.192483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:59.192483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:59.192483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:59.192483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:59.192483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:59.192483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:59.192483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:59.192483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:59.192483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:59.192483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:59.192483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:59.192483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:59.192483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:59.193425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:59.193425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:59.193425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:59.193425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:59.193425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:59.193425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:59.193425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:59.193425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:59.193425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:59.193425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:59.193425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:59.193425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:59.193425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:59.193425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:59.193425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:59.193425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:59.193425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:59.194329 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:59.194329 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:59.194329 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:59.194329 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:59.194329 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 336/649] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:59.194594 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0013eab-4893-441d-958c-fee45f6a7cd8 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:59.251236 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:59.258617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a45dc-bf4b-426b-a6ae-920b89e51c96 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:59.295196 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e1f1e-3940-4fbf-a544-ed4b29b0d103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:59.295943 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e1f1e-3940-4fbf-a544-ed4b29b0d103 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:59.296510 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e1f1e-3940-4fbf-a544-ed4b29b0d103 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:59.296510 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e1f1e-3940-4fbf-a544-ed4b29b0d103 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:59.318126 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0013eab-4893-441d-958c-fee45f6a7cd8 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:59.346073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:59.360005 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13370897-f619-4e9a-9dd3-7ab12c51336b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:59.360889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13370897-f619-4e9a-9dd3-7ab12c51336b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:59.361172 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13370897-f619-4e9a-9dd3-7ab12c51336b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:59.361447 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13370897-f619-4e9a-9dd3-7ab12c51336b None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:59.362218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13370897-f619-4e9a-9dd3-7ab12c51336b None admin] RBAC: Adding query filter params (name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:59.362520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13370897-f619-4e9a-9dd3-7ab12c51336b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:59.363881 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13370897-f619-4e9a-9dd3-7ab12c51336b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:59.372139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 337/650] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:34:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:59.376834 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13370897-f619-4e9a-9dd3-7ab12c51336b None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:59.385817 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e1f1e-3940-4fbf-a544-ed4b29b0d103 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:59.387363 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e1f1e-3940-4fbf-a544-ed4b29b0d103 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:59.485336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13370897-f619-4e9a-9dd3-7ab12c51336b None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:59.496033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:59.502898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 314/651] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:34:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:34:59.503688 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e1f1e-3940-4fbf-a544-ed4b29b0d103 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:59.523752 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775efec-7e26-4e19-82c6-cc10b208fd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:59.524433 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775efec-7e26-4e19-82c6-cc10b208fd48 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:59.524716 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775efec-7e26-4e19-82c6-cc10b208fd48 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:59.524993 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775efec-7e26-4e19-82c6-cc10b208fd48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:59.525861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775efec-7e26-4e19-82c6-cc10b208fd48 None admin] RBAC: Adding query filter params (role.id=805851682adf4debbe99b1681c9fcdfc, scope.project.id=7be83fb6f20341139147373837320148, user.id=1c8f6262cf24419daf803ae71aed24f5) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:59.526624 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775efec-7e26-4e19-82c6-cc10b208fd48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:59.528053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775efec-7e26-4e19-82c6-cc10b208fd48 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:59.541700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 338/652] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:34:59 2022] GET /identity/v3/role_assignments?user.id=1c8f6262cf24419daf803ae71aed24f5&scope.project.id=7be83fb6f20341139147373837320148&role.id=805851682adf4debbe99b1681c9fcdfc => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:59.541700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775efec-7e26-4e19-82c6-cc10b208fd48 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:59.592968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:59.633267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e1f1e-3940-4fbf-a544-ed4b29b0d103 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:59.680784 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad4bce-0aad-4f66-b413-69f89b18758c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:59.681621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad4bce-0aad-4f66-b413-69f89b18758c None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:59.681871 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad4bce-0aad-4f66-b413-69f89b18758c None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:59.682149 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad4bce-0aad-4f66-b413-69f89b18758c None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:59.682907 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad4bce-0aad-4f66-b413-69f89b18758c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:34:59.683219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad4bce-0aad-4f66-b413-69f89b18758c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:59.684651 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad4bce-0aad-4f66-b413-69f89b18758c None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:59.686046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775efec-7e26-4e19-82c6-cc10b208fd48 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:59.702331 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 315/653] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:34:59.703317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad4bce-0aad-4f66-b413-69f89b18758c None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:59.738398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62784d52-2b35-45a6-89b3-5079feb1ed09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:59.740390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62784d52-2b35-45a6-89b3-5079feb1ed09 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:59.740390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62784d52-2b35-45a6-89b3-5079feb1ed09 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:59.740390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62784d52-2b35-45a6-89b3-5079feb1ed09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:59.759418 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62784d52-2b35-45a6-89b3-5079feb1ed09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:59.761195 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62784d52-2b35-45a6-89b3-5079feb1ed09 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:59.780227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-62784d52-2b35-45a6-89b3-5079feb1ed09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:59.780227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:59.780227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:59.780227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:59.780227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:59.780227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:59.780227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:59.780227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:59.780227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:59.780227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:59.780227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:59.780227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:59.780227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:59.780227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:59.780227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:59.781142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:59.781142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:59.781142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:59.781142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:59.781142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:59.781142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:59.781142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:59.781142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:59.781142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:59.781142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:59.781142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:59.781142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:59.781142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:59.781142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:59.781142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:59.781142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:59.781142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:59.782083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:59.782083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:59.782083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:59.782083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:59.784362 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 339/654] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:59.785251 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62784d52-2b35-45a6-89b3-5079feb1ed09 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:59.814174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:59.848488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad4bce-0aad-4f66-b413-69f89b18758c None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:59.893899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7881b55-9259-448b-b6cd-14e19d6e75a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:59.894899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7881b55-9259-448b-b6cd-14e19d6e75a5 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:59.895309 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7881b55-9259-448b-b6cd-14e19d6e75a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:59.895680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7881b55-9259-448b-b6cd-14e19d6e75a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:59.896904 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62784d52-2b35-45a6-89b3-5079feb1ed09 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:34:59.907623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7881b55-9259-448b-b6cd-14e19d6e75a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:59.909310 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7881b55-9259-448b-b6cd-14e19d6e75a5 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:59.929156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-b7881b55-9259-448b-b6cd-14e19d6e75a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:59.929156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:59.929156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:59.929156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:59.929156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:59.929156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:59.929156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:59.929156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:59.929156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:59.929156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:59.929156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:59.929156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:59.929156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:34:59.929156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:34:59.929156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:34:59.930334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:34:59.930334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:59.930334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:59.930334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:59.930334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:59.930334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:59.930334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:59.930334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:59.930334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:59.930334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:59.930334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:59.930334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:59.930334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:59.930334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:59.930334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:59.930334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:59.930334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:59.932478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:34:59.932478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:34:59.932478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:34:59.932478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:34:59.933740 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 316/655] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:34:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:59.933856 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbcb41-149e-40fd-9b10-d3b5f1711f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:34:59.934624 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbcb41-149e-40fd-9b10-d3b5f1711f50 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:34:59.934863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7881b55-9259-448b-b6cd-14e19d6e75a5 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:34:59.934978 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbcb41-149e-40fd-9b10-d3b5f1711f50 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:34:59.935176 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbcb41-149e-40fd-9b10-d3b5f1711f50 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:34:59.938020 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:34:59.949078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cbcb41-149e-40fd-9b10-d3b5f1711f50 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:34:59.951739 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cbcb41-149e-40fd-9b10-d3b5f1711f50 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:34:59.970046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-84cbcb41-149e-40fd-9b10-d3b5f1711f50 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 27 15:34:59.970046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:34:59.970046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:34:59.970046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:34:59.970046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:34:59.970046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:34:59.970046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:34:59.970046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:34:59.970046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:34:59.970046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:34:59.970046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:34:59.970046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:34:59.970046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:34:59.970046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:34:59.970046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:34:59.970046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:34:59.971053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:34:59.971053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:34:59.971053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:34:59.971053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:59.971053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:34:59.971053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:34:59.971053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:34:59.971053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:34:59.971053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:34:59.971053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:34:59.971053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:34:59.971053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:34:59.971053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:34:59.971053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:34:59.971053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:34:59.971053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:34:59.971053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:34:59.971053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:34:59.971963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:34:59.971963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:34:59.971963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:34:59.971963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:34:59.971963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:34:59.971963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:34:59.971963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:34:59.971963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:34:59.971963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 27 15:34:59.971963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:34:59.975007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 340/656] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:34:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:34:59.976127 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbcb41-149e-40fd-9b10-d3b5f1711f50 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:00.083210 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7881b55-9259-448b-b6cd-14e19d6e75a5 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:00.086615 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:00.110361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbcb41-149e-40fd-9b10-d3b5f1711f50 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:00.124875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fab54-fc7b-4898-af65-1155eccc5cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:00.125561 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fab54-fc7b-4898-af65-1155eccc5cba None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:00.125874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fab54-fc7b-4898-af65-1155eccc5cba None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:00.126149 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fab54-fc7b-4898-af65-1155eccc5cba None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:00.126877 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fab54-fc7b-4898-af65-1155eccc5cba None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:00.127193 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fab54-fc7b-4898-af65-1155eccc5cba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:00.128676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fab54-fc7b-4898-af65-1155eccc5cba None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:00.141868 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 317/657] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:00.142519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fab54-fc7b-4898-af65-1155eccc5cba None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:00.152946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47ef8a-0e81-4159-8943-543ba2a4ad13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:00.154411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47ef8a-0e81-4159-8943-543ba2a4ad13 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:00.154967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47ef8a-0e81-4159-8943-543ba2a4ad13 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:00.155455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47ef8a-0e81-4159-8943-543ba2a4ad13 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:00.156496 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47ef8a-0e81-4159-8943-543ba2a4ad13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:00.157023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47ef8a-0e81-4159-8943-543ba2a4ad13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:00.158641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47ef8a-0e81-4159-8943-543ba2a4ad13 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:00.178342 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 341/658] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:00.179564 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47ef8a-0e81-4159-8943-543ba2a4ad13 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:00.190412 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:00.250873 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fab54-fc7b-4898-af65-1155eccc5cba None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:00.286436 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5dea6c-c875-4f93-99fa-7cacc7e58b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:00.287077 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5dea6c-c875-4f93-99fa-7cacc7e58b92 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:00.287360 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5dea6c-c875-4f93-99fa-7cacc7e58b92 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:00.287635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5dea6c-c875-4f93-99fa-7cacc7e58b92 None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:00.288475 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5dea6c-c875-4f93-99fa-7cacc7e58b92 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:00.289030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5dea6c-c875-4f93-99fa-7cacc7e58b92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:00.291154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5dea6c-c875-4f93-99fa-7cacc7e58b92 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:00.297417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47ef8a-0e81-4159-8943-543ba2a4ad13 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:00.309767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 318/659] 10.222.0.20 () {56 vars in 1137 bytes} [Mon Jun 27 15:35:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:00.310528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5dea6c-c875-4f93-99fa-7cacc7e58b92 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:00.338114 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ec176-11bf-4bd3-b202-44289ad060c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:00.339377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ec176-11bf-4bd3-b202-44289ad060c2 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:00.339878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ec176-11bf-4bd3-b202-44289ad060c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:00.340429 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ec176-11bf-4bd3-b202-44289ad060c2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:00.349300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ec176-11bf-4bd3-b202-44289ad060c2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:00.351133 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ec176-11bf-4bd3-b202-44289ad060c2 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:00.370364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-7c0ec176-11bf-4bd3-b202-44289ad060c2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 15:35:00.370364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:00.370364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:00.370364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:00.370364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:00.370364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:00.370364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:00.370364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:00.370364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:00.370364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:00.370364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:00.370364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:00.370364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:35:00.370364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:35:00.370364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:35:00.370364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:35:00.371815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:00.371815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:00.371815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:00.371815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:00.371815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:00.371815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:00.371815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:00.371815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:00.371815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:00.371815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:00.371815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:00.371815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:00.371815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:00.371815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:00.371815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:00.371815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:00.371815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:00.371815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:00.376263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:00.376263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:00.376263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:35:00.376263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:35:00.376263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:35:00.376263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:35:00.376263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:35:00.376263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:35:00.376263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 15:35:00.376263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:00.376263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 342/660] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:35:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:00.376263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ec176-11bf-4bd3-b202-44289ad060c2 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:00.382681 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:00.443759 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:00.450504 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5dea6c-c875-4f93-99fa-7cacc7e58b92 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:00.488774 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ec176-11bf-4bd3-b202-44289ad060c2 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:00.499847 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619394e2-ac02-4a41-9a67-e13f2feccb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:00.500667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619394e2-ac02-4a41-9a67-e13f2feccb60 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:00.501276 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619394e2-ac02-4a41-9a67-e13f2feccb60 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:00.501560 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619394e2-ac02-4a41-9a67-e13f2feccb60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:00.509447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619394e2-ac02-4a41-9a67-e13f2feccb60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:00.511004 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619394e2-ac02-4a41-9a67-e13f2feccb60 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:00.526280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-619394e2-ac02-4a41-9a67-e13f2feccb60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:00.526280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:00.526280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:00.526280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:00.526280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:00.526280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:00.526280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:00.526280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:00.526280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:00.526280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:00.526280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:00.526280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:00.526280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:00.526280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:00.526280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:00.527204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:00.527204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:00.527204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:00.527204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:00.527204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:00.527204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:00.527204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:00.527204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:00.527204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:00.527204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:00.527204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:00.527204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:00.527204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:00.527204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:00.527204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:00.527204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:00.527204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:00.528057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:00.528057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:00.528057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:00.528057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:00.528057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:00.528057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:00.528057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:00.528057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:00.528549 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a75a90-470b-4fe3-b20a-5d2dd60035b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:00.528549 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a75a90-470b-4fe3-b20a-5d2dd60035b4 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:00.528549 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a75a90-470b-4fe3-b20a-5d2dd60035b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:00.528766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 319/661] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:00.528766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619394e2-ac02-4a41-9a67-e13f2feccb60 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:00.532100 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a75a90-470b-4fe3-b20a-5d2dd60035b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:00.605870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a75a90-470b-4fe3-b20a-5d2dd60035b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:00.607252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a75a90-470b-4fe3-b20a-5d2dd60035b4 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:00.610952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:00.635883 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619394e2-ac02-4a41-9a67-e13f2feccb60 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:00.672060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ebeba-3e92-4b20-8ca3-6a61f5234927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:00.673083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ebeba-3e92-4b20-8ca3-6a61f5234927 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:00.673400 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ebeba-3e92-4b20-8ca3-6a61f5234927 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:00.673700 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ebeba-3e92-4b20-8ca3-6a61f5234927 None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:00.674550 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ebeba-3e92-4b20-8ca3-6a61f5234927 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:00.674872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ebeba-3e92-4b20-8ca3-6a61f5234927 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:00.676307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ebeba-3e92-4b20-8ca3-6a61f5234927 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:00.695281 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 320/662] 10.222.0.20 () {56 vars in 1139 bytes} [Mon Jun 27 15:35:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:00.697252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fd7bd-0468-4958-9bb9-e5e4d1492f8a None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:00.697704 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fd7bd-0468-4958-9bb9-e5e4d1492f8a None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:00.697992 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fd7bd-0468-4958-9bb9-e5e4d1492f8a None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:00.703528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 321/663] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:00.704323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fd7bd-0468-4958-9bb9-e5e4d1492f8a None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:00.724930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 343/664] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:35:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:35:00.724930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a75a90-470b-4fe3-b20a-5d2dd60035b4 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:00.726413 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:00.831078 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fd7bd-0468-4958-9bb9-e5e4d1492f8a None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:00.837041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a75a90-470b-4fe3-b20a-5d2dd60035b4 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:00.840501 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:00.869395 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa17648-adf8-4720-9090-35ca304dfe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:00.870018 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa17648-adf8-4720-9090-35ca304dfe4c None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:00.870301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa17648-adf8-4720-9090-35ca304dfe4c None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:00.870579 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa17648-adf8-4720-9090-35ca304dfe4c None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:00.871369 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa17648-adf8-4720-9090-35ca304dfe4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:00.871683 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa17648-adf8-4720-9090-35ca304dfe4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:00.873158 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa17648-adf8-4720-9090-35ca304dfe4c None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:00.875965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f7a9b4-a31e-4c96-8138-00de614b495c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:00.876354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7a9b4-a31e-4c96-8138-00de614b495c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:00.876632 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7a9b4-a31e-4c96-8138-00de614b495c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:00.876906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7a9b4-a31e-4c96-8138-00de614b495c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:00.891506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 322/665] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:00.891506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ce38b-040d-45a1-98a8-481cae05bc30 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:00.891506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ce38b-040d-45a1-98a8-481cae05bc30 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:00.891506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ce38b-040d-45a1-98a8-481cae05bc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:00.905364 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-661ce38b-040d-45a1-98a8-481cae05bc30 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:00.906521 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-661ce38b-040d-45a1-98a8-481cae05bc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:00.955690 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f7a9b4-a31e-4c96-8138-00de614b495c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:00.957409 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f7a9b4-a31e-4c96-8138-00de614b495c None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:00.981677 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 323/666] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:35:00.981677 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ce38b-040d-45a1-98a8-481cae05bc30 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:00.990614 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:01.086105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 344/667] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:35:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:35:01.090664 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ce38b-040d-45a1-98a8-481cae05bc30 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:01.091928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f7a9b4-a31e-4c96-8138-00de614b495c None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:01.095187 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:01.127156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d31dc-deff-4dde-aacc-a47f362b12f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:01.127875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d31dc-deff-4dde-aacc-a47f362b12f4 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:01.128166 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d31dc-deff-4dde-aacc-a47f362b12f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:01.128443 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d31dc-deff-4dde-aacc-a47f362b12f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:01.141196 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1d31dc-deff-4dde-aacc-a47f362b12f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:01.142691 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1d31dc-deff-4dde-aacc-a47f362b12f4 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:01.162292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-0d1d31dc-deff-4dde-aacc-a47f362b12f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:01.162292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:01.162292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:01.162292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:01.162292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:01.162292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:01.162292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:01.162292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:01.162292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:01.162292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:01.162292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:01.162292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:01.162292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:01.162292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:01.162292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:01.163248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:01.163248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:01.163248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:01.163248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:01.163248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:01.163248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:01.163248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:01.163248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:01.163248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:01.163248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:01.163248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:01.163248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:01.163248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:01.163248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:01.163248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:01.163248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:01.163248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:01.164418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:01.164418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:01.164418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:01.164418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:01.164418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 324/668] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:01.166346 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d31dc-deff-4dde-aacc-a47f362b12f4 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:01.203210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f7a9b4-a31e-4c96-8138-00de614b495c None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:01.231178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:01.238657 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deba3d7-c7fd-457e-bf79-41c0c3a43592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:01.239317 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deba3d7-c7fd-457e-bf79-41c0c3a43592 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:01.239597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deba3d7-c7fd-457e-bf79-41c0c3a43592 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:01.239869 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deba3d7-c7fd-457e-bf79-41c0c3a43592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:01.240696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deba3d7-c7fd-457e-bf79-41c0c3a43592 None admin] RBAC: Adding query filter params (role.id=805851682adf4debbe99b1681c9fcdfc, scope.project.id=7be83fb6f20341139147373837320148, user.id=51404afd35b94c47ba5c44f0deb73e30) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:01.241005 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deba3d7-c7fd-457e-bf79-41c0c3a43592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:01.242469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deba3d7-c7fd-457e-bf79-41c0c3a43592 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:01.248989 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 345/669] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:35:01 2022] GET /identity/v3/role_assignments?user.id=51404afd35b94c47ba5c44f0deb73e30&scope.project.id=7be83fb6f20341139147373837320148&role.id=805851682adf4debbe99b1681c9fcdfc => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:01.249734 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deba3d7-c7fd-457e-bf79-41c0c3a43592 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:01.283696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d31dc-deff-4dde-aacc-a47f362b12f4 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:01.334869 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ed9ee-757b-44ae-8725-4dc2365e1cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:01.337246 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ed9ee-757b-44ae-8725-4dc2365e1cbf None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:01.338000 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ed9ee-757b-44ae-8725-4dc2365e1cbf None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:01.338773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ed9ee-757b-44ae-8725-4dc2365e1cbf None admin] PATH_INFO: `/v3/roles/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:01.355027 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ed9ee-757b-44ae-8725-4dc2365e1cbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:01.365796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-8e5ed9ee-757b-44ae-8725-4dc2365e1cbf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:35:01.365796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:01.365796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:01.365796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:01.365796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:01.365796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:01.365796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:01.365796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:01.365796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:01.365796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:01.365796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:01.365796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:01.365796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:01.365796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:01.365796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:01.365796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:01.366794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:01.366794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:01.366794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:01.366794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:01.366794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:01.366794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:01.366794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:01.366794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:01.366794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:01.366794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:01.366794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:01.366794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:01.366794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:01.366794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:01.366794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:01.366794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:01.367612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:01.367612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:01.367612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:01.367612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:01.367612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:01.367612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:01.367612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:01.367612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:01.367612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:01.367612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:01.367612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:35:01.367612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:01.375695 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:01.375889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 325/670] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:35:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:01.375889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ed9ee-757b-44ae-8725-4dc2365e1cbf None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:01.390392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deba3d7-c7fd-457e-bf79-41c0c3a43592 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:01.437373 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df8699-1dc3-4855-a8db-215fc09cfa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:01.438211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df8699-1dc3-4855-a8db-215fc09cfa01 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:01.438498 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df8699-1dc3-4855-a8db-215fc09cfa01 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:01.438773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df8699-1dc3-4855-a8db-215fc09cfa01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:01.451698 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df8699-1dc3-4855-a8db-215fc09cfa01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:01.453839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df8699-1dc3-4855-a8db-215fc09cfa01 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:01.478166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-e4df8699-1dc3-4855-a8db-215fc09cfa01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:01.478166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:01.478166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:01.478166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:01.478166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:01.478166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:01.478166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:01.478166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:01.478166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:01.478166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:01.478166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:01.478166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:01.478166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:01.478166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:01.478166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:01.479135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:01.479135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:01.479135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:01.479135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:01.479135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:01.479135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:01.479135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:01.479135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:01.479135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:01.479135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:01.479135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:01.479135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:01.479135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:01.479135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:01.479135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:01.479135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:01.479135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:01.480000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:01.480000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:01.480000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:01.480000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:01.480000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 346/671] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:01.480741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df8699-1dc3-4855-a8db-215fc09cfa01 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:01.504923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ed9ee-757b-44ae-8725-4dc2365e1cbf None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:01.534617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:01.556003 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d0946-da4e-42b8-b1b2-e2e53456a877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:01.557159 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d0946-da4e-42b8-b1b2-e2e53456a877 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:01.557469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d0946-da4e-42b8-b1b2-e2e53456a877 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:01.557777 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d0946-da4e-42b8-b1b2-e2e53456a877 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:01.558848 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d0946-da4e-42b8-b1b2-e2e53456a877 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:01.559570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d0946-da4e-42b8-b1b2-e2e53456a877 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:01.561301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d0946-da4e-42b8-b1b2-e2e53456a877 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:01.581797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 326/672] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:01.582685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d0946-da4e-42b8-b1b2-e2e53456a877 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:01.610293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df8699-1dc3-4855-a8db-215fc09cfa01 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:01.637704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:01.657329 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a9bec-b74e-4e98-8af5-811a0c018161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:01.658234 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a9bec-b74e-4e98-8af5-811a0c018161 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:01.658527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a9bec-b74e-4e98-8af5-811a0c018161 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:01.658809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a9bec-b74e-4e98-8af5-811a0c018161 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:01.659742 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a9bec-b74e-4e98-8af5-811a0c018161 None admin] RBAC: Adding query filter params (name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:01.660056 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a9bec-b74e-4e98-8af5-811a0c018161 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:01.662264 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a9bec-b74e-4e98-8af5-811a0c018161 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:01.673518 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 347/673] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:35:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:01.676057 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a9bec-b74e-4e98-8af5-811a0c018161 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:01.699713 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d0946-da4e-42b8-b1b2-e2e53456a877 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:01.737402 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de202d-2a28-4ffa-883d-01cfc4aee337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:01.738205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de202d-2a28-4ffa-883d-01cfc4aee337 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:01.738486 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de202d-2a28-4ffa-883d-01cfc4aee337 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:01.738760 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de202d-2a28-4ffa-883d-01cfc4aee337 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:01.739484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de202d-2a28-4ffa-883d-01cfc4aee337 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:01.739788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de202d-2a28-4ffa-883d-01cfc4aee337 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:01.741375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de202d-2a28-4ffa-883d-01cfc4aee337 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:01.755131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 327/674] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:01.756127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de202d-2a28-4ffa-883d-01cfc4aee337 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:01.804389 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:01.805681 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a9bec-b74e-4e98-8af5-811a0c018161 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:01.852172 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7778142-01eb-4b0f-9c14-22371a7d085a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:01.853724 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7778142-01eb-4b0f-9c14-22371a7d085a None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:01.853972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7778142-01eb-4b0f-9c14-22371a7d085a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:01.854398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7778142-01eb-4b0f-9c14-22371a7d085a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:01.863890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7778142-01eb-4b0f-9c14-22371a7d085a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:01.866127 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7778142-01eb-4b0f-9c14-22371a7d085a None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:01.870559 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de202d-2a28-4ffa-883d-01cfc4aee337 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:01.885589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-b7778142-01eb-4b0f-9c14-22371a7d085a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:01.885589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:01.885589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:01.885589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:01.885589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:01.885589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:01.885589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:01.885589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:01.885589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:01.885589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:01.885589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:01.885589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:01.885589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:01.885589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:01.885589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:01.886527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:01.886527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:01.886527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:01.886527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:01.886527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:01.886527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:01.886527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:01.886527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:01.886527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:01.886527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:01.886527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:01.886527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:01.886527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:01.886527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:01.886527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:01.886527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:01.886527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:01.887411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:01.887411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:01.887411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:01.887411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:01.887411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:01.887411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:01.887411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:01.887411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:01.888508 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 348/675] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:01.889872 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9396f5-c4c2-4833-a7b4-e3f827f0509f None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:01.890208 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9396f5-c4c2-4833-a7b4-e3f827f0509f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:01.890495 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9396f5-c4c2-4833-a7b4-e3f827f0509f None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:01.892047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 349/676] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:01.892746 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9396f5-c4c2-4833-a7b4-e3f827f0509f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:01.911886 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615e142-cf97-40a3-a6ba-362b65803ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:01.912599 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615e142-cf97-40a3-a6ba-362b65803ffd None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:01.912882 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615e142-cf97-40a3-a6ba-362b65803ffd None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:01.913355 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615e142-cf97-40a3-a6ba-362b65803ffd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:01.925487 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615e142-cf97-40a3-a6ba-362b65803ffd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:01.926988 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615e142-cf97-40a3-a6ba-362b65803ffd None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:01.943620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:01.947545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-0615e142-cf97-40a3-a6ba-362b65803ffd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:01.947545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:01.947545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:01.947545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:01.947545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:01.947545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:01.947545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:01.947545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:01.947545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:01.947545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:01.947545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:01.947545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:01.947545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:01.947545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:01.947545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:01.948426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:01.948426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:01.948426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:01.948426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:01.948426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:01.948426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:01.948426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:01.948426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:01.948426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:01.948426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:01.948426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:01.948426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:01.948426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:01.948426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:01.948426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:01.948426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:01.948426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:01.949282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:01.949282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:01.949282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:01.949282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:01.949282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 328/677] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:01.949575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615e142-cf97-40a3-a6ba-362b65803ffd None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:02.011208 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9396f5-c4c2-4833-a7b4-e3f827f0509f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:02.049696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076691a-49ac-492b-bdc4-5d2fb99ba889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:02.050474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076691a-49ac-492b-bdc4-5d2fb99ba889 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:02.050765 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076691a-49ac-492b-bdc4-5d2fb99ba889 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:02.051035 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076691a-49ac-492b-bdc4-5d2fb99ba889 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:02.054728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:02.059848 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076691a-49ac-492b-bdc4-5d2fb99ba889 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:02.061320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076691a-49ac-492b-bdc4-5d2fb99ba889 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:02.066550 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615e142-cf97-40a3-a6ba-362b65803ffd None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:02.077877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-7076691a-49ac-492b-bdc4-5d2fb99ba889 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:02.077877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:02.077877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:02.077877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:02.077877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:02.077877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:02.077877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:02.077877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:02.077877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:02.077877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:02.077877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:02.077877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:02.077877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:02.077877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:02.077877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:02.078792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:02.078792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:02.078792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:02.078792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:02.078792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:02.078792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:02.078792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:02.078792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:02.078792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:02.078792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:02.078792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:02.078792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:02.078792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:02.078792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:02.078792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:02.078792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:02.078792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:02.079671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:02.079671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:02.079671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:02.079671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:02.079671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:02.079671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:02.079671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:02.079671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:02.079671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 350/678] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:02.080931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7f31b-893a-4936-9cbb-f6090c6c7a73 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:02.080931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7f31b-893a-4936-9cbb-f6090c6c7a73 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:02.081171 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7f31b-893a-4936-9cbb-f6090c6c7a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:02.100085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-a8f7f31b-893a-4936-9cbb-f6090c6c7a73 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:02.101355 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-a8f7f31b-893a-4936-9cbb-f6090c6c7a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:02.108407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122112e-ee82-4eb7-b74b-613a4fb20d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:02.109075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122112e-ee82-4eb7-b74b-613a4fb20d5d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:02.109356 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122112e-ee82-4eb7-b74b-613a4fb20d5d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:02.109627 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122112e-ee82-4eb7-b74b-613a4fb20d5d None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:02.110448 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7122112e-ee82-4eb7-b74b-613a4fb20d5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:02.110753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7122112e-ee82-4eb7-b74b-613a4fb20d5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:02.112236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7122112e-ee82-4eb7-b74b-613a4fb20d5d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:02.125848 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 329/679] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:02.126684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122112e-ee82-4eb7-b74b-613a4fb20d5d None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:02.189750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 351/680] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:35:02.189750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7f31b-893a-4936-9cbb-f6090c6c7a73 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:02.215628 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:02.254173 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122112e-ee82-4eb7-b74b-613a4fb20d5d None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:02.291004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7f31b-893a-4936-9cbb-f6090c6c7a73 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:02.292253 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafafce-eec0-4765-9b1f-d4d4c633c427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:02.292923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafafce-eec0-4765-9b1f-d4d4c633c427 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:02.293208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafafce-eec0-4765-9b1f-d4d4c633c427 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:02.293486 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafafce-eec0-4765-9b1f-d4d4c633c427 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:02.294392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fafafce-eec0-4765-9b1f-d4d4c633c427 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:02.294567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fafafce-eec0-4765-9b1f-d4d4c633c427 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:02.295954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fafafce-eec0-4765-9b1f-d4d4c633c427 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:02.308318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 330/681] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:02.309076 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafafce-eec0-4765-9b1f-d4d4c633c427 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:02.326035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:02.333142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f6e23-741d-4117-9b13-4d8adb16c5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:02.334159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f6e23-741d-4117-9b13-4d8adb16c5d2 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:02.334491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f6e23-741d-4117-9b13-4d8adb16c5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:02.334772 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f6e23-741d-4117-9b13-4d8adb16c5d2 None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:02.335555 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f6e23-741d-4117-9b13-4d8adb16c5d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:02.335865 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f6e23-741d-4117-9b13-4d8adb16c5d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:02.337728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f6e23-741d-4117-9b13-4d8adb16c5d2 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:02.350093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 352/682] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:02.350917 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f6e23-741d-4117-9b13-4d8adb16c5d2 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:02.432491 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafafce-eec0-4765-9b1f-d4d4c633c427 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:02.457045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:02.467340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f6e23-741d-4117-9b13-4d8adb16c5d2 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:02.471819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4afd33-3bd3-4b15-b5b5-e69a29396497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:02.472670 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4afd33-3bd3-4b15-b5b5-e69a29396497 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:02.473102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4afd33-3bd3-4b15-b5b5-e69a29396497 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:02.473541 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4afd33-3bd3-4b15-b5b5-e69a29396497 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:02.474802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4afd33-3bd3-4b15-b5b5-e69a29396497 None admin] RBAC: Adding query filter params (role.id=805851682adf4debbe99b1681c9fcdfc, scope.project.id=7be83fb6f20341139147373837320148, user.id=493fcde4a45944efbc0b52c44e61b411) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:02.475124 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4afd33-3bd3-4b15-b5b5-e69a29396497 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:02.477027 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4afd33-3bd3-4b15-b5b5-e69a29396497 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:02.494116 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 331/683] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:35:02 2022] GET /identity/v3/role_assignments?user.id=493fcde4a45944efbc0b52c44e61b411&scope.project.id=7be83fb6f20341139147373837320148&role.id=805851682adf4debbe99b1681c9fcdfc => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:02.494116 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4afd33-3bd3-4b15-b5b5-e69a29396497 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:02.518358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15eedf6-d159-4a71-9b40-cb4d6714e047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:02.519732 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15eedf6-d159-4a71-9b40-cb4d6714e047 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:02.520014 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15eedf6-d159-4a71-9b40-cb4d6714e047 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:02.520327 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15eedf6-d159-4a71-9b40-cb4d6714e047 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:02.537705 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15eedf6-d159-4a71-9b40-cb4d6714e047 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:02.549366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-f15eedf6-d159-4a71-9b40-cb4d6714e047 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:35:02.549366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:02.549366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:02.549366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:02.549366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:02.549366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:02.549366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:02.549366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:02.549366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:02.549366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:02.549366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:02.549366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:02.549366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:02.549366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:02.549366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:02.549366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:02.550401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:02.550401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:02.550401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:02.550401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:02.550401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:02.550401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:02.550401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:02.550401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:02.550401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:02.550401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:02.550401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:02.550401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:02.550401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:02.550401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:02.550401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:02.550401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:02.551226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:02.551226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:02.551226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:02.551226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:02.551226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:02.551226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:02.551226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:02.551226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:02.551226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:02.551226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:02.551226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:35:02.551226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:02.551853 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 353/684] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:35:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:02.553721 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15eedf6-d159-4a71-9b40-cb4d6714e047 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:02.601000 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:02.631520 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4afd33-3bd3-4b15-b5b5-e69a29396497 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:02.676667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc48fd-b08e-4349-b8ff-4cc9836b18ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:02.678162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc48fd-b08e-4349-b8ff-4cc9836b18ef None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:02.678804 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc48fd-b08e-4349-b8ff-4cc9836b18ef None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:02.679278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc48fd-b08e-4349-b8ff-4cc9836b18ef None admin] PATH_INFO: `/v3/users/swift` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:02.691127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cc48fd-b08e-4349-b8ff-4cc9836b18ef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:02.693203 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cc48fd-b08e-4349-b8ff-4cc9836b18ef None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:02.697144 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15eedf6-d159-4a71-9b40-cb4d6714e047 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:02.714500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-38cc48fd-b08e-4349-b8ff-4cc9836b18ef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:35:02.714500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:02.714500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:02.714500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:02.714500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:02.714500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:02.714500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:02.714500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:02.714500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:02.714500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:02.714500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:02.714500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:02.714500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:35:02.714500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:35:02.714500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:35:02.714500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:35:02.715531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:02.715531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:02.715531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:02.715531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:02.715531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:02.715531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:02.715531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:02.715531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:02.715531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:02.715531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:02.715531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:02.715531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:02.715531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:02.715531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:02.715531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:02.715531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:02.715531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:02.715531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:02.716456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:02.716456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:02.716456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:35:02.716456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:35:02.716456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:35:02.716456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:35:02.716456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:35:02.716456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:35:02.716456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:35:02.716456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:02.718694 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 332/685] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:35:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:02.719703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc48fd-b08e-4349-b8ff-4cc9836b18ef None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:02.744193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8e53a-86d9-48c3-b9b9-0e230aabd644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:02.744859 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8e53a-86d9-48c3-b9b9-0e230aabd644 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:02.745201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8e53a-86d9-48c3-b9b9-0e230aabd644 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:02.745485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8e53a-86d9-48c3-b9b9-0e230aabd644 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:02.746374 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8e53a-86d9-48c3-b9b9-0e230aabd644 None admin] RBAC: Adding query filter params (role.id=805851682adf4debbe99b1681c9fcdfc, scope.project.id=7be83fb6f20341139147373837320148, user.id=42dee639e2834d99b7468f911c37cf8f) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:02.747064 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8e53a-86d9-48c3-b9b9-0e230aabd644 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:02.748335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8e53a-86d9-48c3-b9b9-0e230aabd644 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:02.755687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 354/686] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:35:02 2022] GET /identity/v3/role_assignments?user.id=42dee639e2834d99b7468f911c37cf8f&scope.project.id=7be83fb6f20341139147373837320148&role.id=805851682adf4debbe99b1681c9fcdfc => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:02.761663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8e53a-86d9-48c3-b9b9-0e230aabd644 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:02.785806 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:02.849931 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc48fd-b08e-4349-b8ff-4cc9836b18ef None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:02.891636 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8e53a-86d9-48c3-b9b9-0e230aabd644 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:02.937664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7f59c-d68f-4f05-b6d5-eccc4a1bd94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:02.937664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7f59c-d68f-4f05-b6d5-eccc4a1bd94d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:02.937664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7f59c-d68f-4f05-b6d5-eccc4a1bd94d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:02.937664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7f59c-d68f-4f05-b6d5-eccc4a1bd94d None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:02.938110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7f59c-d68f-4f05-b6d5-eccc4a1bd94d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:02.938110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7f59c-d68f-4f05-b6d5-eccc4a1bd94d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:02.938110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7f59c-d68f-4f05-b6d5-eccc4a1bd94d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:02.947581 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330863e-a484-45a2-a87e-4e4e32ebff09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:02.948652 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330863e-a484-45a2-a87e-4e4e32ebff09 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:02.949132 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330863e-a484-45a2-a87e-4e4e32ebff09 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:02.949614 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4330863e-a484-45a2-a87e-4e4e32ebff09 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:02.950592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330863e-a484-45a2-a87e-4e4e32ebff09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:02.951093 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330863e-a484-45a2-a87e-4e4e32ebff09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:02.952703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4330863e-a484-45a2-a87e-4e4e32ebff09 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:02.955582 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:02.957224 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 355/687] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Jun 27 15:35:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:02.963180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7f59c-d68f-4f05-b6d5-eccc4a1bd94d None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:02.965955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 333/688] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:35:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:02.978074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330863e-a484-45a2-a87e-4e4e32ebff09 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:03.076313 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:03.110796 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7f59c-d68f-4f05-b6d5-eccc4a1bd94d None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:03.126120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4330863e-a484-45a2-a87e-4e4e32ebff09 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:03.158901 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e4d72-97a2-4c69-a34b-f2a2946764f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:03.159945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e4d72-97a2-4c69-a34b-f2a2946764f4 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:03.160422 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e4d72-97a2-4c69-a34b-f2a2946764f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:03.160990 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e4d72-97a2-4c69-a34b-f2a2946764f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:03.181548 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3200c-03be-4c74-be7f-e65bab7dda22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:03.182552 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3200c-03be-4c74-be7f-e65bab7dda22 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:03.182839 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3200c-03be-4c74-be7f-e65bab7dda22 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:03.183118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3200c-03be-4c74-be7f-e65bab7dda22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:03.202042 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3200c-03be-4c74-be7f-e65bab7dda22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:03.204060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3200c-03be-4c74-be7f-e65bab7dda22 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:03.226925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-55d3200c-03be-4c74-be7f-e65bab7dda22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:03.226925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:03.226925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:03.226925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:03.226925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:03.226925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:03.226925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:03.226925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:03.226925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:03.226925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:03.226925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:03.226925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:03.226925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:03.226925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:03.226925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:03.227952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:03.227952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:03.227952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:03.227952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:03.227952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:03.227952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:03.227952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:03.227952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:03.227952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:03.227952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:03.227952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:03.227952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:03.227952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:03.227952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:03.227952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:03.227952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:03.227952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:03.228838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:03.228838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:03.228838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:03.228838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:03.228838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 334/689] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:03.234336 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3200c-03be-4c74-be7f-e65bab7dda22 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:03.260111 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e4d72-97a2-4c69-a34b-f2a2946764f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:03.261619 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e4d72-97a2-4c69-a34b-f2a2946764f4 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:03.337986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:03.385999 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:03.416327 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 356/690] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:35:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:35:03.421855 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e4d72-97a2-4c69-a34b-f2a2946764f4 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:03.482640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3200c-03be-4c74-be7f-e65bab7dda22 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:03.530378 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e4d72-97a2-4c69-a34b-f2a2946764f4 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:03.534407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c0702-c4c2-410f-92cc-4974a2749977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:03.535102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c0702-c4c2-410f-92cc-4974a2749977 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:03.535392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c0702-c4c2-410f-92cc-4974a2749977 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:03.535670 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c0702-c4c2-410f-92cc-4974a2749977 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:03.536415 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c0702-c4c2-410f-92cc-4974a2749977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:03.536726 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c0702-c4c2-410f-92cc-4974a2749977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:03.550415 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c0702-c4c2-410f-92cc-4974a2749977 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:03.568968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cc30c-22e7-45db-8ffb-52e518e20e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:03.569814 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cc30c-22e7-45db-8ffb-52e518e20e72 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:03.570102 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cc30c-22e7-45db-8ffb-52e518e20e72 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:03.570386 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cc30c-22e7-45db-8ffb-52e518e20e72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:03.582756 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0cc30c-22e7-45db-8ffb-52e518e20e72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:03.584236 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0cc30c-22e7-45db-8ffb-52e518e20e72 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:03.596057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 335/691] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:03.606689 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c0702-c4c2-410f-92cc-4974a2749977 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:03.610189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-ee0cc30c-22e7-45db-8ffb-52e518e20e72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:03.610189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:03.610189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:03.610189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:03.610189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:03.610189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:03.610189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:03.610189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:03.610189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:03.610189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:03.610189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:03.610189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:03.610189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:03.610189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:03.610189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:03.611148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:03.611148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:03.611148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:03.611148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:03.611148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:03.611148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:03.611148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:03.611148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:03.611148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:03.611148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:03.611148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:03.611148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:03.611148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:03.611148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:03.611148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:03.611148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:03.611148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:03.612010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:03.612010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:03.612010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:03.612010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:03.612010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 357/692] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:03.619079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cc30c-22e7-45db-8ffb-52e518e20e72 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:03.649405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:03.745809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cc30c-22e7-45db-8ffb-52e518e20e72 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:03.751310 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c0702-c4c2-410f-92cc-4974a2749977 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:03.763663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:03.783533 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd2251-7959-43a3-8de3-a576d3aba14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:03.784550 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd2251-7959-43a3-8de3-a576d3aba14e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:03.785069 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd2251-7959-43a3-8de3-a576d3aba14e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:03.785541 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd2251-7959-43a3-8de3-a576d3aba14e None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:03.786513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd2251-7959-43a3-8de3-a576d3aba14e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:03.787020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd2251-7959-43a3-8de3-a576d3aba14e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:03.788602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd2251-7959-43a3-8de3-a576d3aba14e None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:03.791698 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b95246-4235-4fe3-af6a-7f4b170152f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:03.792519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b95246-4235-4fe3-af6a-7f4b170152f9 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:03.792804 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b95246-4235-4fe3-af6a-7f4b170152f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:03.793083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b95246-4235-4fe3-af6a-7f4b170152f9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:03.801229 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b95246-4235-4fe3-af6a-7f4b170152f9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:03.802737 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b95246-4235-4fe3-af6a-7f4b170152f9 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:03.804269 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 358/693] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:03.805975 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00540f7e-0fbd-460c-880a-55dca165b50b None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:03.806702 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00540f7e-0fbd-460c-880a-55dca165b50b None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:03.807181 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00540f7e-0fbd-460c-880a-55dca165b50b None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:03.808922 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 359/694] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:03.809866 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00540f7e-0fbd-460c-880a-55dca165b50b None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:03.824589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-d2b95246-4235-4fe3-af6a-7f4b170152f9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:35:03.824589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:03.824589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:03.824589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:03.824589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:03.824589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:03.824589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:03.824589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:03.824589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:03.824589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:03.824589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:03.824589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:03.824589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:35:03.824589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:35:03.824589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:35:03.824589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:35:03.825567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:03.825567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:03.825567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:03.825567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:03.825567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:03.825567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:03.825567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:03.825567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:03.825567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:03.825567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:03.825567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:03.825567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:03.825567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:03.825567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:03.825567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:03.825567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:03.825567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:03.825567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:03.826525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:03.826525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:03.826525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:35:03.826525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:35:03.826525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:35:03.826525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:35:03.826525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:35:03.826525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:35:03.826525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:35:03.826525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:03.830753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 336/695] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Jun 27 15:35:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:03.832222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ffba0-dad5-44e9-9ccc-476a547adb46 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:03.832515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ffba0-dad5-44e9-9ccc-476a547adb46 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:03.832794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ffba0-dad5-44e9-9ccc-476a547adb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:03.850514 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-935ffba0-dad5-44e9-9ccc-476a547adb46 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:03.851713 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-935ffba0-dad5-44e9-9ccc-476a547adb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:03.917734 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:03.925724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 337/696] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:35:03.926750 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ffba0-dad5-44e9-9ccc-476a547adb46 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:03.930329 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00540f7e-0fbd-460c-880a-55dca165b50b None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:03.977468 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12c57a-bc37-4e32-adb8-765ad7ca8ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:03.979374 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12c57a-bc37-4e32-adb8-765ad7ca8ff6 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:03.979771 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12c57a-bc37-4e32-adb8-765ad7ca8ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:03.980147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12c57a-bc37-4e32-adb8-765ad7ca8ff6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:03.991301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12c57a-bc37-4e32-adb8-765ad7ca8ff6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:03.995138 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12c57a-bc37-4e32-adb8-765ad7ca8ff6 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:04.015956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-5e12c57a-bc37-4e32-adb8-765ad7ca8ff6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:04.015956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:04.015956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:04.015956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:04.015956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:04.015956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:04.015956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:04.015956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:04.015956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:04.015956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:04.015956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:04.015956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:04.015956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:04.015956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:04.015956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:04.017008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:04.017008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:04.017008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:04.017008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:04.017008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:04.017008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:04.017008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:04.017008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:04.017008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:04.017008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:04.017008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:04.017008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:04.017008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:04.017008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:04.017008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:04.017008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:04.017008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:04.017947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:04.017947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:04.017947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:04.017947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:04.017947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:04.017947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:04.017947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:04.017947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:04.018847 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 360/697] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:04.022390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12c57a-bc37-4e32-adb8-765ad7ca8ff6 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:04.047485 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:04.050846 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ffba0-dad5-44e9-9ccc-476a547adb46 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:04.100213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc698a-f1e9-432d-a68b-040b456b8505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:04.101002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc698a-f1e9-432d-a68b-040b456b8505 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:04.102193 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc698a-f1e9-432d-a68b-040b456b8505 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:04.103154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc698a-f1e9-432d-a68b-040b456b8505 None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:04.104429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc698a-f1e9-432d-a68b-040b456b8505 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:04.104878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc698a-f1e9-432d-a68b-040b456b8505 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:04.106456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc698a-f1e9-432d-a68b-040b456b8505 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:04.131189 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 338/698] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Jun 27 15:35:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:04.132265 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc698a-f1e9-432d-a68b-040b456b8505 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:04.143861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12c57a-bc37-4e32-adb8-765ad7ca8ff6 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:04.183192 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe17add8-b39b-445d-9916-bbc46274184e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:04.184030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17add8-b39b-445d-9916-bbc46274184e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:04.184315 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17add8-b39b-445d-9916-bbc46274184e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:04.184592 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17add8-b39b-445d-9916-bbc46274184e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:04.185263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe17add8-b39b-445d-9916-bbc46274184e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:04.186724 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe17add8-b39b-445d-9916-bbc46274184e None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:04.202453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-fe17add8-b39b-445d-9916-bbc46274184e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 15:35:04.202453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:04.202453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:04.202453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:04.202453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:04.202453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:04.202453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:04.202453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:04.202453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 15:35:04.202453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:04.202453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:04.202453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:04.202453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:04.202453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:04.202453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:04.202453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:04.203469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:04.203469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:04.203469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:04.203469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:04.203469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:04.203469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:04.203469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:04.203469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:04.203469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:04.203469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:04.203469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:04.203469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:04.203469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:04.203469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:04.203469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:04.203469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:04.204314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:04.204314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:04.204314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:04.204314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:04.204314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:04.204314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:04.204314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:04.204314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:04.204314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:04.204314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:04.204314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:04.204314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:04.204314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:04.204314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 15:35:04.204314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:04.204314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 361/699] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Jun 27 15:35:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:04.205145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe17add8-b39b-445d-9916-bbc46274184e None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:04.219523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:04.257384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc698a-f1e9-432d-a68b-040b456b8505 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:04.296559 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f851be5-0f9e-4678-bc6e-ebc920ec71f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:04.297476 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f851be5-0f9e-4678-bc6e-ebc920ec71f7 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:04.298044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f851be5-0f9e-4678-bc6e-ebc920ec71f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:04.298475 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f851be5-0f9e-4678-bc6e-ebc920ec71f7 None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:04.299500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f851be5-0f9e-4678-bc6e-ebc920ec71f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:04.299992 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f851be5-0f9e-4678-bc6e-ebc920ec71f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:04.301944 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f851be5-0f9e-4678-bc6e-ebc920ec71f7 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:04.314492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe17add8-b39b-445d-9916-bbc46274184e None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:04.317849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 339/700] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:04.318919 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f851be5-0f9e-4678-bc6e-ebc920ec71f7 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:04.341685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:04.352004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b3441-77be-4553-bdf4-80a54b8eacfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:04.352865 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b3441-77be-4553-bdf4-80a54b8eacfa None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:04.353155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b3441-77be-4553-bdf4-80a54b8eacfa None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:04.353439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b3441-77be-4553-bdf4-80a54b8eacfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:04.431444 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b3441-77be-4553-bdf4-80a54b8eacfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:04.432923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b3441-77be-4553-bdf4-80a54b8eacfa None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:04.437672 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f851be5-0f9e-4678-bc6e-ebc920ec71f7 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:04.445022 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:04.478005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb3cd8-2d40-414b-a4a2-101e6cb8fcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:04.479095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb3cd8-2d40-414b-a4a2-101e6cb8fcb6 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:04.479594 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb3cd8-2d40-414b-a4a2-101e6cb8fcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:04.480062 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb3cd8-2d40-414b-a4a2-101e6cb8fcb6 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:04.481095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb3cd8-2d40-414b-a4a2-101e6cb8fcb6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:04.481846 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb3cd8-2d40-414b-a4a2-101e6cb8fcb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:04.483521 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb3cd8-2d40-414b-a4a2-101e6cb8fcb6 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:04.489820 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 340/701] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Jun 27 15:35:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:04.490999 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb3cd8-2d40-414b-a4a2-101e6cb8fcb6 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:04.545362 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 362/702] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:35:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 15:35:04.545362 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b3441-77be-4553-bdf4-80a54b8eacfa None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:04.593050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:04.615063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb3cd8-2d40-414b-a4a2-101e6cb8fcb6 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:04.648354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b3441-77be-4553-bdf4-80a54b8eacfa None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:04.655584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d50df5-2096-43e1-9a13-5e5d49e003d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:04.656783 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d50df5-2096-43e1-9a13-5e5d49e003d4 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:04.657294 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d50df5-2096-43e1-9a13-5e5d49e003d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:04.657807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d50df5-2096-43e1-9a13-5e5d49e003d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:04.658947 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d50df5-2096-43e1-9a13-5e5d49e003d4 None admin] RBAC: Adding query filter params (role.id=805851682adf4debbe99b1681c9fcdfc, scope.project.id=7be83fb6f20341139147373837320148, user.id=8e83ea69ded040448fe75caa221ad377) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:04.659476 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d50df5-2096-43e1-9a13-5e5d49e003d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:04.660419 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:04.662425 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d50df5-2096-43e1-9a13-5e5d49e003d4 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:04.671362 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 341/703] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:35:04 2022] GET /identity/v3/role_assignments?user.id=8e83ea69ded040448fe75caa221ad377&scope.project.id=7be83fb6f20341139147373837320148&role.id=805851682adf4debbe99b1681c9fcdfc => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:04.672628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d50df5-2096-43e1-9a13-5e5d49e003d4 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:04.707203 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6b859-cc7a-4874-a243-9cfab7ebc59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:04.707788 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6b859-cc7a-4874-a243-9cfab7ebc59c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:04.708074 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6b859-cc7a-4874-a243-9cfab7ebc59c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:04.708349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6b859-cc7a-4874-a243-9cfab7ebc59c None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:04.709082 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6b859-cc7a-4874-a243-9cfab7ebc59c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:04.709402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6b859-cc7a-4874-a243-9cfab7ebc59c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:04.710867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6b859-cc7a-4874-a243-9cfab7ebc59c None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:04.726243 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 363/704] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Jun 27 15:35:04 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:04.808482 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d50df5-2096-43e1-9a13-5e5d49e003d4 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:04.853174 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b0b23-2b4a-456d-8d80-993bdad7aedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:04.854097 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b0b23-2b4a-456d-8d80-993bdad7aedc None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:04.854382 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b0b23-2b4a-456d-8d80-993bdad7aedc None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:04.854660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b0b23-2b4a-456d-8d80-993bdad7aedc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:04.867864 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b0b23-2b4a-456d-8d80-993bdad7aedc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:04.870279 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b0b23-2b4a-456d-8d80-993bdad7aedc None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:04.878123 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:04.893474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-e59b0b23-2b4a-456d-8d80-993bdad7aedc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:04.893474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:04.893474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:04.893474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:04.893474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:04.893474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:04.893474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:04.893474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:04.893474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:04.893474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:04.893474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:04.893474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:04.893474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:04.893474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:04.893474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:04.894452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:04.894452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:04.894452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:04.894452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:04.894452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:04.894452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:04.894452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:04.894452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:04.894452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:04.894452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:04.894452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:04.894452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:04.894452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:04.894452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:04.894452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:04.894452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:04.894452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:04.895800 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:04.895800 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:04.895800 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:04.895800 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:04.895800 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 342/705] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:04.901336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6b859-cc7a-4874-a243-9cfab7ebc59c None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:05.029272 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6b859-cc7a-4874-a243-9cfab7ebc59c None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:05.070676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08bbd0-d59e-4d07-8b9d-8f1615678e87 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:05.071003 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08bbd0-d59e-4d07-8b9d-8f1615678e87 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:05.071448 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08bbd0-d59e-4d07-8b9d-8f1615678e87 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:05.073092 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afa62b-2c8c-4f17-865f-aa9444a4ff2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:05.073767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 343/706] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:05.073879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afa62b-2c8c-4f17-865f-aa9444a4ff2f None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:05.074082 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afa62b-2c8c-4f17-865f-aa9444a4ff2f None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:05.074362 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afa62b-2c8c-4f17-865f-aa9444a4ff2f None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:05.075161 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04afa62b-2c8c-4f17-865f-aa9444a4ff2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:05.075474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04afa62b-2c8c-4f17-865f-aa9444a4ff2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:05.076860 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04afa62b-2c8c-4f17-865f-aa9444a4ff2f None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:05.079214 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1043d16-c710-4653-96f9-72e314eb315d None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:05.079750 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1043d16-c710-4653-96f9-72e314eb315d None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:05.080232 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1043d16-c710-4653-96f9-72e314eb315d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:05.094581 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 364/707] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:05.101217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afa62b-2c8c-4f17-865f-aa9444a4ff2f None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:05.113899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-b1043d16-c710-4653-96f9-72e314eb315d None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:05.115643 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-b1043d16-c710-4653-96f9-72e314eb315d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:05.140981 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:05.224028 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afa62b-2c8c-4f17-865f-aa9444a4ff2f None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:05.228037 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 344/708] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:35:05.254559 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1043d16-c710-4653-96f9-72e314eb315d None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:05.281410 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf346f-21d0-46ac-9251-7657e523ce33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:05.282593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf346f-21d0-46ac-9251-7657e523ce33 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:05.282880 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf346f-21d0-46ac-9251-7657e523ce33 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:05.283152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf346f-21d0-46ac-9251-7657e523ce33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:05.292978 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf346f-21d0-46ac-9251-7657e523ce33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:05.294470 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf346f-21d0-46ac-9251-7657e523ce33 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:05.311147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-3ddf346f-21d0-46ac-9251-7657e523ce33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:05.311147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:05.311147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:05.311147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:05.311147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:05.311147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:05.311147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:05.311147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:05.311147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:05.311147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:05.311147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:05.311147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:05.311147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:05.311147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:05.311147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:05.312369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:05.312369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:05.312369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:05.312369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:05.312369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:05.312369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:05.312369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:05.312369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:05.312369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:05.312369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:05.312369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:05.312369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:05.312369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:05.312369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:05.312369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:05.312369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:05.312369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:05.313937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:05.313937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:05.313937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:05.313937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:05.313937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:05.313937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:05.313937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:05.313937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:05.313937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 365/709] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:05.314487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0a0fc-47a6-48e4-8399-333170aef566 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:05.314747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0a0fc-47a6-48e4-8399-333170aef566 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:05.315288 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0a0fc-47a6-48e4-8399-333170aef566 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:05.316908 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 366/710] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:05.323000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0a0fc-47a6-48e4-8399-333170aef566 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:05.371065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1043d16-c710-4653-96f9-72e314eb315d None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:05.401344 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:05.421064 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247b351-2b84-4b7b-a171-b799ac66b83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:05.422177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247b351-2b84-4b7b-a171-b799ac66b83d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:05.423092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247b351-2b84-4b7b-a171-b799ac66b83d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:05.423519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247b351-2b84-4b7b-a171-b799ac66b83d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:05.439547 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247b351-2b84-4b7b-a171-b799ac66b83d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:05.441589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:05.442213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247b351-2b84-4b7b-a171-b799ac66b83d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:05.451514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0a0fc-47a6-48e4-8399-333170aef566 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:05.468181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-3247b351-2b84-4b7b-a171-b799ac66b83d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:05.468181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:05.468181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:05.468181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:05.468181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:05.468181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:05.468181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:05.468181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:05.468181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:05.468181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:05.468181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:05.468181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:05.468181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:05.468181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:05.468181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:05.469350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:05.469350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:05.469350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:05.469350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:05.469350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:05.469350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:05.469350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:05.469350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:05.469350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:05.469350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:05.469350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:05.469350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:05.469350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:05.469350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:05.469350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:05.469350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:05.469350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:05.470521 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:05.470521 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:05.470521 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:05.470521 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:05.473238 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 345/711] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:05.475463 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75efb3b-49b9-40bf-9aa8-2d8e1df95e1d None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:05.475906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75efb3b-49b9-40bf-9aa8-2d8e1df95e1d None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:05.476349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75efb3b-49b9-40bf-9aa8-2d8e1df95e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:05.497345 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726cc08-fae7-436c-8d21-d232537212fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:05.498288 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726cc08-fae7-436c-8d21-d232537212fc None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:05.498584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-d75efb3b-49b9-40bf-9aa8-2d8e1df95e1d None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:05.499034 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726cc08-fae7-436c-8d21-d232537212fc None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:05.499034 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0726cc08-fae7-436c-8d21-d232537212fc None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:05.499994 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0726cc08-fae7-436c-8d21-d232537212fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:05.500495 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0726cc08-fae7-436c-8d21-d232537212fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:05.500804 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d75efb3b-49b9-40bf-9aa8-2d8e1df95e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:05.502201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0726cc08-fae7-436c-8d21-d232537212fc None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:05.518547 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 367/712] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:05.519951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726cc08-fae7-436c-8d21-d232537212fc None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:05.586026 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 346/713] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:35:05.587798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75efb3b-49b9-40bf-9aa8-2d8e1df95e1d None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:05.657008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0726cc08-fae7-436c-8d21-d232537212fc None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:05.694516 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8fbb7-27df-4227-8689-fe5b0ea79924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:05.695357 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8fbb7-27df-4227-8689-fe5b0ea79924 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:05.695737 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8fbb7-27df-4227-8689-fe5b0ea79924 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:05.697070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8fbb7-27df-4227-8689-fe5b0ea79924 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:05.697780 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8fbb7-27df-4227-8689-fe5b0ea79924 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:05.698160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8fbb7-27df-4227-8689-fe5b0ea79924 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:05.699630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8fbb7-27df-4227-8689-fe5b0ea79924 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:05.716460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 368/714] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:05.717813 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8fbb7-27df-4227-8689-fe5b0ea79924 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:05.725016 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75efb3b-49b9-40bf-9aa8-2d8e1df95e1d None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:05.737317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:05.740698 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:05.766941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87e16a-2ae9-426a-b478-a262d1486a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:05.767783 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87e16a-2ae9-426a-b478-a262d1486a45 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:05.768793 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87e16a-2ae9-426a-b478-a262d1486a45 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:05.768793 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87e16a-2ae9-426a-b478-a262d1486a45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:05.769563 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87e16a-2ae9-426a-b478-a262d1486a45 None admin] RBAC: Adding query filter params (role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.project.id=7be83fb6f20341139147373837320148, user.id=1c8f6262cf24419daf803ae71aed24f5) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:05.770015 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87e16a-2ae9-426a-b478-a262d1486a45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:05.771693 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87e16a-2ae9-426a-b478-a262d1486a45 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:05.778410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 347/715] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:35:05 2022] GET /identity/v3/role_assignments?user.id=1c8f6262cf24419daf803ae71aed24f5&scope.project.id=7be83fb6f20341139147373837320148&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:05.779563 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87e16a-2ae9-426a-b478-a262d1486a45 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:05.869592 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8fbb7-27df-4227-8689-fe5b0ea79924 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:05.910102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87e16a-2ae9-426a-b478-a262d1486a45 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:05.918234 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a63c8-aa74-4f47-90d6-6957c29f2f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:05.919952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a63c8-aa74-4f47-90d6-6957c29f2f94 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:05.924640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a63c8-aa74-4f47-90d6-6957c29f2f94 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:05.926236 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a63c8-aa74-4f47-90d6-6957c29f2f94 None admin] PATH_INFO: `/v3/services/network` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:05.927161 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a63c8-aa74-4f47-90d6-6957c29f2f94 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:05.928957 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a63c8-aa74-4f47-90d6-6957c29f2f94 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:05.941776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-f01a63c8-aa74-4f47-90d6-6957c29f2f94 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 15:35:05.941776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:05.941776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:05.941776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:05.941776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:05.941776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:05.941776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:05.941776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:05.941776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 15:35:05.941776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:05.941776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:05.941776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:05.941776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:05.941776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:05.941776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:05.941776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:05.941776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:05.942798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:05.942798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:05.942798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:05.942798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:05.942798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:05.942798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:05.942798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:05.942798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:05.942798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:05.942798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:05.942798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:05.942798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:05.942798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:05.942798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:05.942798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:05.942798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:05.943610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:05.943610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:05.943610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:05.943610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:05.943610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:05.943610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:05.943610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:05.943610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:05.943610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:05.943610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:05.943610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:05.943610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:05.943610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 15:35:05.943610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:05.947278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 369/716] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:05 2022] GET /identity/v3/services/network => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:05.953386 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a63c8-aa74-4f47-90d6-6957c29f2f94 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:05.955424 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3674e-c6d3-4190-aab5-abc0fe59f4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:05.956123 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3674e-c6d3-4190-aab5-abc0fe59f4ed None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:05.956404 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3674e-c6d3-4190-aab5-abc0fe59f4ed None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:05.956675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3674e-c6d3-4190-aab5-abc0fe59f4ed None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:05.957397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3674e-c6d3-4190-aab5-abc0fe59f4ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:05.959683 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3674e-c6d3-4190-aab5-abc0fe59f4ed None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:05.998557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 348/717] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jun 27 15:35:05 2022] POST /identity/v3/users => generated 254 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:06.074257 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:06.101515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a63c8-aa74-4f47-90d6-6957c29f2f94 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:06.145074 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99049ec9-d1f2-4323-96c6-3717122265c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:06.145782 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99049ec9-d1f2-4323-96c6-3717122265c3 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:06.146064 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99049ec9-d1f2-4323-96c6-3717122265c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:06.146343 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99049ec9-d1f2-4323-96c6-3717122265c3 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:06.147058 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99049ec9-d1f2-4323-96c6-3717122265c3 None admin] RBAC: Adding query filter params (name=network) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:06.147364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99049ec9-d1f2-4323-96c6-3717122265c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:06.148807 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99049ec9-d1f2-4323-96c6-3717122265c3 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:06.155301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 370/718] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jun 27 15:35:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:06.160903 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3674e-c6d3-4190-aab5-abc0fe59f4ed None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:06.215314 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:06.295114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3674e-c6d3-4190-aab5-abc0fe59f4ed None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:06.332650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317c20d-ce1e-4f01-a40f-8fb1685e7927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:06.333380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317c20d-ce1e-4f01-a40f-8fb1685e7927 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:06.333691 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317c20d-ce1e-4f01-a40f-8fb1685e7927 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:06.333977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317c20d-ce1e-4f01-a40f-8fb1685e7927 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:06.334753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317c20d-ce1e-4f01-a40f-8fb1685e7927 None admin] RBAC: Adding query filter params (type=network) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:06.335068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317c20d-ce1e-4f01-a40f-8fb1685e7927 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:06.336471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317c20d-ce1e-4f01-a40f-8fb1685e7927 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:06.343472 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 349/719] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jun 27 15:35:06 2022] GET /identity/v3/services?type=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:07.318685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85375f-865c-4ec7-b6ce-f81e837f3cab None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:07.319237 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85375f-865c-4ec7-b6ce-f81e837f3cab None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:07.319237 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85375f-865c-4ec7-b6ce-f81e837f3cab None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:07.320834 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 371/720] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:07.326312 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7deb00-bc2e-4ad7-bc66-3d113ec24f13 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:07.327130 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7deb00-bc2e-4ad7-bc66-3d113ec24f13 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:07.327629 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7deb00-bc2e-4ad7-bc66-3d113ec24f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:07.350639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-0c7deb00-bc2e-4ad7-bc66-3d113ec24f13 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:07.352214 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-0c7deb00-bc2e-4ad7-bc66-3d113ec24f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:07.437248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 350/721] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:35:07.472912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85375f-865c-4ec7-b6ce-f81e837f3cab None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:07.498766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c95d4-070b-452b-9d7e-1e21b5795855 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:07.499398 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c95d4-070b-452b-9d7e-1e21b5795855 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:07.499898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c95d4-070b-452b-9d7e-1e21b5795855 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:07.501474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 351/722] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:07.506417 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc855c7-5341-45f8-a9ba-d0f66a206909 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:07.506912 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc855c7-5341-45f8-a9ba-d0f66a206909 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:07.507403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc855c7-5341-45f8-a9ba-d0f66a206909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:07.534775 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-fcc855c7-5341-45f8-a9ba-d0f66a206909 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:07.537471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-fcc855c7-5341-45f8-a9ba-d0f66a206909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:07.582996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:07.638026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85375f-865c-4ec7-b6ce-f81e837f3cab None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:07.690371 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908deed-74eb-4811-afb1-a0bf0ea36055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:07.692588 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908deed-74eb-4811-afb1-a0bf0ea36055 None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:07.694401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908deed-74eb-4811-afb1-a0bf0ea36055 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:07.694885 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908deed-74eb-4811-afb1-a0bf0ea36055 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:07.698011 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908deed-74eb-4811-afb1-a0bf0ea36055 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:07.701748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908deed-74eb-4811-afb1-a0bf0ea36055 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:07.713679 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 352/723] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:35:07.727018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 372/724] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jun 27 15:35:07 2022] POST /identity/v3/services => generated 251 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:07.743533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc855c7-5341-45f8-a9ba-d0f66a206909 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:07.931060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc855c7-5341-45f8-a9ba-d0f66a206909 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:07.986203 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0b6e6-98f4-4813-994a-b626ae363dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:07.987023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0b6e6-98f4-4813-994a-b626ae363dda None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:07.987311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0b6e6-98f4-4813-994a-b626ae363dda None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:07.987591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0b6e6-98f4-4813-994a-b626ae363dda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:08.001500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0b6e6-98f4-4813-994a-b626ae363dda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:08.010144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-89a0b6e6-98f4-4813-994a-b626ae363dda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:35:08.010144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:08.010144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:08.010144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:08.010144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:08.010144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:08.010144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:08.010144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:08.010144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:08.010144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:08.010144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:08.010144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:08.010144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:08.010144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:08.010144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:08.010144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:08.011111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:08.011111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:08.011111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:08.011111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:08.011111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:08.011111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:08.011111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:08.011111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:08.011111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:08.011111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:08.011111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:08.011111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:08.011111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:08.011111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:08.011111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:08.011111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:08.011958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:08.011958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:08.011958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:08.011958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:08.011958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:08.011958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:08.011958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:08.011958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:08.011958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:08.011958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:08.011958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:35:08.011958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:08.013157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 353/725] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:35:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:08.018013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908deed-74eb-4811-afb1-a0bf0ea36055 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:08.024223 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:08.140919 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908deed-74eb-4811-afb1-a0bf0ea36055 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:08.184978 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ffd53-f265-4f1b-be4f-e331be5ae227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:08.186068 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ffd53-f265-4f1b-be4f-e331be5ae227 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:08.186572 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ffd53-f265-4f1b-be4f-e331be5ae227 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:08.187061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ffd53-f265-4f1b-be4f-e331be5ae227 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:08.188017 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ffd53-f265-4f1b-be4f-e331be5ae227 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:08.188525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ffd53-f265-4f1b-be4f-e331be5ae227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:08.190134 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ffd53-f265-4f1b-be4f-e331be5ae227 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:08.200745 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 373/726] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:35:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:08.209669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0b6e6-98f4-4813-994a-b626ae363dda None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:08.230343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:08.341095 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38125353-77b7-4f65-adf1-1c19a7daf1af None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:08.342075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38125353-77b7-4f65-adf1-1c19a7daf1af None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:08.342717 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38125353-77b7-4f65-adf1-1c19a7daf1af None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:08.343914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 374/727] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:08.350154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e92467-2a58-4021-9341-d9f96ee7a5ff None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:08.350444 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e92467-2a58-4021-9341-d9f96ee7a5ff None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:08.350727 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e92467-2a58-4021-9341-d9f96ee7a5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:08.366035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0b6e6-98f4-4813-994a-b626ae363dda None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:08.386734 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-19e92467-2a58-4021-9341-d9f96ee7a5ff None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:08.387952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-19e92467-2a58-4021-9341-d9f96ee7a5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:08.419435 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c02590-290c-4208-92aa-1fa068c482b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:08.420259 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c02590-290c-4208-92aa-1fa068c482b7 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:08.420547 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c02590-290c-4208-92aa-1fa068c482b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:08.420824 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c02590-290c-4208-92aa-1fa068c482b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:08.443126 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c02590-290c-4208-92aa-1fa068c482b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:08.444554 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c02590-290c-4208-92aa-1fa068c482b7 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:08.476667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-74c02590-290c-4208-92aa-1fa068c482b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:08.476667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:08.476667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:08.476667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:08.476667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:08.476667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:08.476667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:08.476667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:08.476667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:08.476667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:08.476667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:08.476667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:08.476667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:08.476667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:08.476667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:08.478048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:08.478048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:08.478048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:08.478048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:08.478048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:08.478048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:08.478048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:08.478048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:08.478048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:08.478048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:08.478048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:08.478048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:08.478048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:08.478048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:08.478048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:08.478048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:08.478048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:08.479115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:08.479115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:08.479115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:08.479115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:08.482191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 354/728] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:08.487464 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c02590-290c-4208-92aa-1fa068c482b7 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:08.496518 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:08.511392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 375/729] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 15:35:08.545390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e92467-2a58-4021-9341-d9f96ee7a5ff None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:08.613360 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:08.642985 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c02590-290c-4208-92aa-1fa068c482b7 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:08.708580 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec1f09-c077-454a-9b0c-1dcedc133c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:08.709184 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec1f09-c077-454a-9b0c-1dcedc133c2c None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:08.709470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec1f09-c077-454a-9b0c-1dcedc133c2c None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:08.709785 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec1f09-c077-454a-9b0c-1dcedc133c2c None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:08.710529 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec1f09-c077-454a-9b0c-1dcedc133c2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:08.710840 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec1f09-c077-454a-9b0c-1dcedc133c2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:08.712880 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbec1f09-c077-454a-9b0c-1dcedc133c2c None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:08.726428 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 355/730] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:08.732642 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec1f09-c077-454a-9b0c-1dcedc133c2c None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:08.746972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e92467-2a58-4021-9341-d9f96ee7a5ff None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:08.797293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397a0b6-263e-43e7-b1b4-32f9030d80de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:08.798255 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397a0b6-263e-43e7-b1b4-32f9030d80de None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:08.798549 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397a0b6-263e-43e7-b1b4-32f9030d80de None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:08.798830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397a0b6-263e-43e7-b1b4-32f9030d80de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:08.816614 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397a0b6-263e-43e7-b1b4-32f9030d80de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:08.828864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-5397a0b6-263e-43e7-b1b4-32f9030d80de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:35:08.828864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:08.828864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:08.828864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:08.828864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:08.828864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:08.828864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:08.828864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:08.828864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:08.828864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:08.828864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:08.828864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:08.828864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:08.828864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:08.828864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:08.828864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:08.829941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:08.829941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:08.829941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:08.829941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:08.829941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:08.829941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:08.829941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:08.829941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:08.829941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:08.829941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:08.829941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:08.829941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:08.829941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:08.829941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:08.829941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:08.829941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:08.830763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:08.830763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:08.830763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:08.830763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:08.830763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:08.830763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:08.830763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:08.830763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:08.830763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:08.830763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:08.830763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:35:08.830763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:08.833723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 376/731] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:35:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:08.835139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c3233-3568-41ca-9330-fefcd2411ebb None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:08.835426 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c3233-3568-41ca-9330-fefcd2411ebb None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:08.835705 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c3233-3568-41ca-9330-fefcd2411ebb None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:08.841795 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 377/732] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:08.842739 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c3233-3568-41ca-9330-fefcd2411ebb None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:08.904019 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbec1f09-c077-454a-9b0c-1dcedc133c2c None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:08.920864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:08.935277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:08.948993 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee35f43-be26-4a3d-a95b-de9280bbd956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:08.949830 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee35f43-be26-4a3d-a95b-de9280bbd956 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:08.950114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee35f43-be26-4a3d-a95b-de9280bbd956 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:08.950385 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee35f43-be26-4a3d-a95b-de9280bbd956 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:08.958355 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee35f43-be26-4a3d-a95b-de9280bbd956 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:08.959894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee35f43-be26-4a3d-a95b-de9280bbd956 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:08.976213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-aee35f43-be26-4a3d-a95b-de9280bbd956 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:35:08.976213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:08.976213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:08.976213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:08.976213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:08.976213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:08.976213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:08.976213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:08.976213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:08.976213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:08.976213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:08.976213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:08.976213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:35:08.976213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:35:08.976213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:35:08.976213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:35:08.977955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:08.977955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:08.977955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:08.977955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:08.977955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:08.977955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:08.977955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:08.977955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:08.977955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:08.977955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:08.977955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:08.977955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:08.977955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:08.977955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:08.977955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:08.977955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:08.977955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:08.977955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:08.979098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:08.979098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:08.979098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:35:08.979098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:35:08.979098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:35:08.979098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:35:08.979098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:35:08.979098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:35:08.979098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:35:08.979098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:08.982858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 356/733] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:35:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:08.984318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d87a6-e45b-4233-9a7a-6029b77b4578 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:08.984608 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d87a6-e45b-4233-9a7a-6029b77b4578 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:08.984852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c3233-3568-41ca-9330-fefcd2411ebb None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:08.984969 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d87a6-e45b-4233-9a7a-6029b77b4578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:09.001946 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-ea5d87a6-e45b-4233-9a7a-6029b77b4578 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:09.003316 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-ea5d87a6-e45b-4233-9a7a-6029b77b4578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:09.029482 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842deab1-5683-4cc9-be36-eab82757e2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:09.030992 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842deab1-5683-4cc9-be36-eab82757e2f8 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:09.031432 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842deab1-5683-4cc9-be36-eab82757e2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:09.031883 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842deab1-5683-4cc9-be36-eab82757e2f8 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:09.033226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842deab1-5683-4cc9-be36-eab82757e2f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:09.033710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842deab1-5683-4cc9-be36-eab82757e2f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:09.035631 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842deab1-5683-4cc9-be36-eab82757e2f8 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:09.051333 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 378/734] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:35:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:09.051333 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842deab1-5683-4cc9-be36-eab82757e2f8 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:09.088876 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 357/735] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 15:35:09.088876 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d87a6-e45b-4233-9a7a-6029b77b4578 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:09.166557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842deab1-5683-4cc9-be36-eab82757e2f8 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:09.185236 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:09.204312 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d87a6-e45b-4233-9a7a-6029b77b4578 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:09.209286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a6927-d901-4ca4-8da3-4ed96c13590f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:09.210066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a6927-d901-4ca4-8da3-4ed96c13590f None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:09.210296 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a6927-d901-4ca4-8da3-4ed96c13590f None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:09.210569 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a6927-d901-4ca4-8da3-4ed96c13590f None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:09.211538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a6927-d901-4ca4-8da3-4ed96c13590f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:09.211893 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a6927-d901-4ca4-8da3-4ed96c13590f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:09.213336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3a6927-d901-4ca4-8da3-4ed96c13590f None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:09.219571 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:09.232728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 379/736] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Jun 27 15:35:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:09.232728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3261e-a422-438e-b968-353c77ef21e4 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:09.232728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3261e-a422-438e-b968-353c77ef21e4 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:09.232728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3261e-a422-438e-b968-353c77ef21e4 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:09.232728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 380/737] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:09.238219 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3261e-a422-438e-b968-353c77ef21e4 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:09.255130 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ff8d9-08e1-46dc-9df9-6593e729695b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:09.256209 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ff8d9-08e1-46dc-9df9-6593e729695b None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:09.256495 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ff8d9-08e1-46dc-9df9-6593e729695b None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:09.256775 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ff8d9-08e1-46dc-9df9-6593e729695b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:09.269690 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ff8d9-08e1-46dc-9df9-6593e729695b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:09.270774 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ff8d9-08e1-46dc-9df9-6593e729695b None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:09.290621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-c83ff8d9-08e1-46dc-9df9-6593e729695b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:09.290621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:09.290621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:09.290621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:09.290621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:09.290621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:09.290621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:09.290621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:09.290621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:09.290621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:09.290621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:09.290621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:09.290621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:09.290621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:09.290621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:09.291917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:09.291917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:09.291917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:09.291917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:09.291917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:09.291917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:09.291917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:09.291917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:09.291917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:09.291917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:09.291917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:09.291917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:09.291917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:09.291917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:09.291917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:09.291917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:09.291917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:09.292966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:09.292966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:09.292966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:09.292966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:09.296320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 358/738] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:09.297203 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ff8d9-08e1-46dc-9df9-6593e729695b None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:09.359535 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3261e-a422-438e-b968-353c77ef21e4 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:09.396580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc1914-57f2-48f3-a531-68f3f9a8a9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:09.397251 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc1914-57f2-48f3-a531-68f3f9a8a9ed None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:09.397536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc1914-57f2-48f3-a531-68f3f9a8a9ed None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:09.397848 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc1914-57f2-48f3-a531-68f3f9a8a9ed None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:09.398571 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc1914-57f2-48f3-a531-68f3f9a8a9ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:09.399965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc1914-57f2-48f3-a531-68f3f9a8a9ed None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:09.409454 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ff8d9-08e1-46dc-9df9-6593e729695b None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:09.414623 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 381/739] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jun 27 15:35:09 2022] POST /identity/v3/services => generated 240 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:09.415719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2a3ed-7bfa-488e-88d1-3bb1cc7d4f2a None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:09.416007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2a3ed-7bfa-488e-88d1-3bb1cc7d4f2a None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:09.416288 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d2a3ed-7bfa-488e-88d1-3bb1cc7d4f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:09.437777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-b0d2a3ed-7bfa-488e-88d1-3bb1cc7d4f2a None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:09.439094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-b0d2a3ed-7bfa-488e-88d1-3bb1cc7d4f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:09.470387 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381e029-7294-42c9-80c3-0d31d16d6107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:09.471739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381e029-7294-42c9-80c3-0d31d16d6107 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:09.473027 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:09.473306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381e029-7294-42c9-80c3-0d31d16d6107 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:09.474780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381e029-7294-42c9-80c3-0d31d16d6107 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:09.524495 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:09.533662 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 382/740] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:09.533662 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2a3ed-7bfa-488e-88d1-3bb1cc7d4f2a None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:09.601449 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381e029-7294-42c9-80c3-0d31d16d6107 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:09.603269 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381e029-7294-42c9-80c3-0d31d16d6107 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:09.668553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d2a3ed-7bfa-488e-88d1-3bb1cc7d4f2a None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:09.715260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b746662-0783-4119-885c-60ac708c8c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:09.716610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b746662-0783-4119-885c-60ac708c8c87 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:09.717112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b746662-0783-4119-885c-60ac708c8c87 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:09.717909 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b746662-0783-4119-885c-60ac708c8c87 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:09.719280 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b746662-0783-4119-885c-60ac708c8c87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:09.719811 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b746662-0783-4119-885c-60ac708c8c87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:09.721482 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b746662-0783-4119-885c-60ac708c8c87 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:09.738829 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 383/741] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:09.739601 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 359/742] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:35:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:35:09.740504 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b746662-0783-4119-885c-60ac708c8c87 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:09.746689 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381e029-7294-42c9-80c3-0d31d16d6107 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:09.774254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:09.835969 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:09.872631 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b746662-0783-4119-885c-60ac708c8c87 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:09.902119 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381e029-7294-42c9-80c3-0d31d16d6107 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:09.919631 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f30f5f-2115-4ef2-bdd3-bd17946c5606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:09.920456 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f30f5f-2115-4ef2-bdd3-bd17946c5606 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:09.920740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f30f5f-2115-4ef2-bdd3-bd17946c5606 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:09.921020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f30f5f-2115-4ef2-bdd3-bd17946c5606 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:09.934238 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f30f5f-2115-4ef2-bdd3-bd17946c5606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:09.942467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-54f30f5f-2115-4ef2-bdd3-bd17946c5606 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:35:09.942467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:09.942467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:09.942467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:09.942467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:09.942467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:09.942467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:09.942467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:09.942467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:09.942467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:09.942467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:09.942467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:09.942467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:09.942467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:09.942467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:09.942467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:09.943451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:09.943451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:09.943451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:09.943451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:09.943451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:09.943451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:09.943451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:09.943451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:09.943451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:09.943451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:09.943451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:09.943451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:09.943451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:09.943451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:09.943451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:09.943451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:09.945370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:09.945370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:09.945370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:09.945370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:09.945370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:09.945370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:09.945370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:09.945370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:09.945370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:09.945370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:09.945370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:35:09.945370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:09.945370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 384/743] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:35:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:09.945370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f30f5f-2115-4ef2-bdd3-bd17946c5606 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:09.954139 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813022d-ea33-481a-b30a-2b2de84f1263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:09.955323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813022d-ea33-481a-b30a-2b2de84f1263 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:09.955823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813022d-ea33-481a-b30a-2b2de84f1263 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:09.956363 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813022d-ea33-481a-b30a-2b2de84f1263 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:09.966352 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813022d-ea33-481a-b30a-2b2de84f1263 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:09.968181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813022d-ea33-481a-b30a-2b2de84f1263 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:09.984996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-c813022d-ea33-481a-b30a-2b2de84f1263 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:35:09.984996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:09.984996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:09.984996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:09.984996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:09.984996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:09.984996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:09.984996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:09.984996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:09.984996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:09.984996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:09.984996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:09.984996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:35:09.984996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:35:09.984996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:35:09.984996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:35:09.986447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:09.986447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:09.986447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:09.986447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:09.986447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:09.986447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:09.986447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:09.986447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:09.986447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:09.986447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:09.986447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:09.986447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:09.986447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:09.986447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:09.986447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:09.986447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:09.986447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:09.986447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:09.987384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:09.987384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:09.987384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:35:09.987384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:35:09.987384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:35:09.987384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:35:09.987384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:35:09.987384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:35:09.987384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:35:09.987384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:09.990766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 360/744] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Jun 27 15:35:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:09.991828 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813022d-ea33-481a-b30a-2b2de84f1263 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:10.049404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:10.068422 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f30f5f-2115-4ef2-bdd3-bd17946c5606 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:10.109321 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813022d-ea33-481a-b30a-2b2de84f1263 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:10.109808 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88dabd-2794-4099-bdd7-0f6e5b633b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:10.110954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88dabd-2794-4099-bdd7-0f6e5b633b0d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:10.111423 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88dabd-2794-4099-bdd7-0f6e5b633b0d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:10.111999 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88dabd-2794-4099-bdd7-0f6e5b633b0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:10.125169 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88dabd-2794-4099-bdd7-0f6e5b633b0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:10.127279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88dabd-2794-4099-bdd7-0f6e5b633b0d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:10.133781 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:10.152105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-7f88dabd-2794-4099-bdd7-0f6e5b633b0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:10.152105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:10.152105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:10.152105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:10.152105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:10.152105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:10.152105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:10.152105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:10.152105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:10.152105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:10.152105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:10.152105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:10.152105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:10.152105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:10.152105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:10.153227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:10.153227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:10.153227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:10.153227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:10.153227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:10.153227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:10.153227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:10.153227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:10.153227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:10.153227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:10.153227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:10.153227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:10.153227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:10.153227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:10.153227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:10.153227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:10.153227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:10.154323 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:10.154323 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:10.154323 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:10.154323 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:10.155860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3e4a8-b97d-4fd6-8268-654245c34648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:10.156083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 385/745] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:10.156970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3e4a8-b97d-4fd6-8268-654245c34648 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:10.157224 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88dabd-2794-4099-bdd7-0f6e5b633b0d None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:10.157553 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3e4a8-b97d-4fd6-8268-654245c34648 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:10.158010 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3e4a8-b97d-4fd6-8268-654245c34648 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:10.158942 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3e4a8-b97d-4fd6-8268-654245c34648 None admin] RBAC: Adding query filter params (name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:10.159400 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3e4a8-b97d-4fd6-8268-654245c34648 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:10.161159 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3e4a8-b97d-4fd6-8268-654245c34648 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:10.172138 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 361/746] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:35:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:10.173338 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3e4a8-b97d-4fd6-8268-654245c34648 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:10.293673 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88dabd-2794-4099-bdd7-0f6e5b633b0d None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:10.301419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3e4a8-b97d-4fd6-8268-654245c34648 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:10.333571 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e7bbe-583c-43b5-838b-8b8023e1cb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:10.334268 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e7bbe-583c-43b5-838b-8b8023e1cb47 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:10.334577 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e7bbe-583c-43b5-838b-8b8023e1cb47 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:10.334817 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e7bbe-583c-43b5-838b-8b8023e1cb47 None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:10.335672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e7bbe-583c-43b5-838b-8b8023e1cb47 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:10.335989 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e7bbe-583c-43b5-838b-8b8023e1cb47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:10.338777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e7bbe-583c-43b5-838b-8b8023e1cb47 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:10.342481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:10.351935 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218e532-1a10-46e3-9270-1886c4174554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:10.352719 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218e532-1a10-46e3-9270-1886c4174554 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:10.353017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218e532-1a10-46e3-9270-1886c4174554 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:10.353302 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218e532-1a10-46e3-9270-1886c4174554 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:10.354071 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218e532-1a10-46e3-9270-1886c4174554 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:10.354383 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218e532-1a10-46e3-9270-1886c4174554 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:10.355783 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218e532-1a10-46e3-9270-1886c4174554 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:10.361290 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 386/747] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Jun 27 15:35:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:10.361989 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e7bbe-583c-43b5-838b-8b8023e1cb47 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:10.379236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 362/748] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:10.381002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4bfa2-cdc4-45bc-a01f-45521cdfe3f2 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:10.381002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4bfa2-cdc4-45bc-a01f-45521cdfe3f2 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:10.381317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4bfa2-cdc4-45bc-a01f-45521cdfe3f2 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:10.384270 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 363/749] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:10.384270 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4bfa2-cdc4-45bc-a01f-45521cdfe3f2 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:10.441485 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:10.486170 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e7bbe-583c-43b5-838b-8b8023e1cb47 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:10.515297 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4bfa2-cdc4-45bc-a01f-45521cdfe3f2 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:10.528687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aead1c-4357-47c4-85f7-a79df302100e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:10.529491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aead1c-4357-47c4-85f7-a79df302100e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:10.529834 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aead1c-4357-47c4-85f7-a79df302100e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:10.530191 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aead1c-4357-47c4-85f7-a79df302100e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:10.548488 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aead1c-4357-47c4-85f7-a79df302100e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:10.550715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aead1c-4357-47c4-85f7-a79df302100e None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:10.556572 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133839c-a216-4165-b3e2-9678e0d2b598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:10.557369 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133839c-a216-4165-b3e2-9678e0d2b598 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:10.557676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133839c-a216-4165-b3e2-9678e0d2b598 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:10.557957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133839c-a216-4165-b3e2-9678e0d2b598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:10.568382 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133839c-a216-4165-b3e2-9678e0d2b598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:10.569862 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133839c-a216-4165-b3e2-9678e0d2b598 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:10.576976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-22aead1c-4357-47c4-85f7-a79df302100e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:10.576976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:10.576976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:10.576976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:10.576976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:10.576976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:10.576976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:10.576976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:10.576976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:10.576976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:10.576976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:10.576976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:10.576976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:10.576976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:10.576976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:10.577915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:10.577915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:10.577915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:10.577915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:10.577915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:10.577915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:10.577915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:10.577915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:10.577915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:10.577915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:10.577915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:10.577915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:10.577915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:10.577915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:10.577915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:10.577915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:10.577915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:10.578852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:10.578852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:10.578852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:10.578852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:10.578852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 387/750] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:10.580531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aead1c-4357-47c4-85f7-a79df302100e None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:10.593837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-f133839c-a216-4165-b3e2-9678e0d2b598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:10.593837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:10.593837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:10.593837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:10.593837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:10.593837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:10.593837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:10.593837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:10.593837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:10.593837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:10.593837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:10.593837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:10.593837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:10.593837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:10.593837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:10.595055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:10.595055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:10.595055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:10.595055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:10.595055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:10.595055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:10.595055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:10.595055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:10.595055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:10.595055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:10.595055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:10.595055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:10.595055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:10.595055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:10.595055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:10.595055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:10.595055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:10.595952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:10.595952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:10.595952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:10.595952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:10.595952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 364/751] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:10.597337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddfc4c-ca70-4bb4-bfc4-c1a33afe000c None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:10.597337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddfc4c-ca70-4bb4-bfc4-c1a33afe000c None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:10.597337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddfc4c-ca70-4bb4-bfc4-c1a33afe000c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:10.617363 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-6eddfc4c-ca70-4bb4-bfc4-c1a33afe000c None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:10.618652 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-6eddfc4c-ca70-4bb4-bfc4-c1a33afe000c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:10.651015 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:10.702911 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 365/752] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:10.706624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eddfc4c-ca70-4bb4-bfc4-c1a33afe000c None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:10.725332 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aead1c-4357-47c4-85f7-a79df302100e None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:10.737407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:10.770379 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4593808-325e-49cd-8bf0-936fbed46dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:10.771176 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4593808-325e-49cd-8bf0-936fbed46dd8 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:10.771462 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4593808-325e-49cd-8bf0-936fbed46dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:10.771740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4593808-325e-49cd-8bf0-936fbed46dd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:10.779922 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4593808-325e-49cd-8bf0-936fbed46dd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:10.781578 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4593808-325e-49cd-8bf0-936fbed46dd8 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:10.798128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-f4593808-325e-49cd-8bf0-936fbed46dd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:10.798128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:10.798128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:10.798128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:10.798128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:10.798128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:10.798128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:10.798128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:10.798128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:10.798128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:10.798128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:10.798128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:10.798128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:10.798128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:10.798128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:10.799108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:10.799108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:10.799108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:10.799108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:10.799108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:10.799108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:10.799108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:10.799108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:10.799108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:10.799108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:10.799108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:10.799108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:10.799108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:10.799108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:10.799108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:10.799108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:10.799108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:10.799979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:10.799979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:10.799979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:10.799979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:10.799979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:10.799979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:10.799979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:10.799979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:10.799979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 388/753] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:10.800980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4593808-325e-49cd-8bf0-936fbed46dd8 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:10.839267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eddfc4c-ca70-4bb4-bfc4-c1a33afe000c None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:10.887613 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602abd9-7127-4bf1-a9d2-188c1224a2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:10.888438 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602abd9-7127-4bf1-a9d2-188c1224a2b0 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:10.888983 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602abd9-7127-4bf1-a9d2-188c1224a2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:10.889434 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602abd9-7127-4bf1-a9d2-188c1224a2b0 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:10.890248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602abd9-7127-4bf1-a9d2-188c1224a2b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:10.890565 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602abd9-7127-4bf1-a9d2-188c1224a2b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:10.891967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602abd9-7127-4bf1-a9d2-188c1224a2b0 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:10.904652 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 366/754] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:10.905607 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602abd9-7127-4bf1-a9d2-188c1224a2b0 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:10.944306 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4593808-325e-49cd-8bf0-936fbed46dd8 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:10.952250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:10.983589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c610c-3ea1-4258-9615-a57b0895d6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:10.984282 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c610c-3ea1-4258-9615-a57b0895d6e3 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:10.984569 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c610c-3ea1-4258-9615-a57b0895d6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:10.984842 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c610c-3ea1-4258-9615-a57b0895d6e3 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:10.985570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c610c-3ea1-4258-9615-a57b0895d6e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:10.985901 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c610c-3ea1-4258-9615-a57b0895d6e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:10.987280 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c610c-3ea1-4258-9615-a57b0895d6e3 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:11.003316 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 389/755] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:11.007254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c610c-3ea1-4258-9615-a57b0895d6e3 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:11.017101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:11.035391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602abd9-7127-4bf1-a9d2-188c1224a2b0 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:11.081941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0f494-20f4-478d-b3d5-8be7ef8ca27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:11.082768 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0f494-20f4-478d-b3d5-8be7ef8ca27e None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:11.083293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0f494-20f4-478d-b3d5-8be7ef8ca27e None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:11.083293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0f494-20f4-478d-b3d5-8be7ef8ca27e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:11.083964 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0f494-20f4-478d-b3d5-8be7ef8ca27e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:11.085462 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0f494-20f4-478d-b3d5-8be7ef8ca27e None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:11.101073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-43e0f494-20f4-478d-b3d5-8be7ef8ca27e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 15:35:11.101073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:11.101073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:11.101073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:11.101073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:11.101073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:11.101073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:11.101073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:11.101073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 15:35:11.101073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:11.101073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:11.101073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:11.101073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:11.101073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:11.101073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:11.101073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:11.102909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:11.102909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:11.102909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:11.102909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:11.102909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:11.102909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:11.102909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:11.102909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:11.102909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:11.102909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:11.102909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:11.102909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:11.102909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:11.102909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:11.102909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:11.102909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:11.103810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:11.103810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:11.103810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:11.103810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:11.103810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:11.103810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:11.103810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:11.103810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:11.103810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:11.103810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:11.103810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:11.103810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:11.103810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:11.103810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 15:35:11.103810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:11.103810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 367/756] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Jun 27 15:35:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:11.107437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0f494-20f4-478d-b3d5-8be7ef8ca27e None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:11.133161 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c610c-3ea1-4258-9615-a57b0895d6e3 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:11.177274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c4570a-6644-4d2f-9760-2aeb2cd97fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:11.178567 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4570a-6644-4d2f-9760-2aeb2cd97fb0 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:11.179021 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4570a-6644-4d2f-9760-2aeb2cd97fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:11.179448 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4570a-6644-4d2f-9760-2aeb2cd97fb0 None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:11.180473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c4570a-6644-4d2f-9760-2aeb2cd97fb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:11.180946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c4570a-6644-4d2f-9760-2aeb2cd97fb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:11.183812 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c4570a-6644-4d2f-9760-2aeb2cd97fb0 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:11.204111 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 390/757] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:11.205512 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c4570a-6644-4d2f-9760-2aeb2cd97fb0 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:11.243746 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:11.264686 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0f494-20f4-478d-b3d5-8be7ef8ca27e None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:11.304174 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadc62e-af66-4300-bdba-e6b44ded5758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:11.305001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadc62e-af66-4300-bdba-e6b44ded5758 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:11.305289 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadc62e-af66-4300-bdba-e6b44ded5758 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:11.306177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadc62e-af66-4300-bdba-e6b44ded5758 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:11.314984 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadc62e-af66-4300-bdba-e6b44ded5758 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:11.316467 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadc62e-af66-4300-bdba-e6b44ded5758 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:11.335558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-fcadc62e-af66-4300-bdba-e6b44ded5758 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:35:11.335558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:11.335558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:11.335558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:11.335558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:11.335558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:11.335558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:11.335558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:11.335558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:11.335558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:11.335558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:11.335558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:11.335558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:35:11.335558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:35:11.335558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:35:11.335558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:35:11.336630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:11.336630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:11.336630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:11.336630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:11.336630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:11.336630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:11.336630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:11.336630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:11.336630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:11.336630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:11.336630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:11.336630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:11.336630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:11.336630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:11.336630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:11.336630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:11.336630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:11.336630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:11.339267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:11.339267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:11.339267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:35:11.339267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:35:11.339267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:35:11.339267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:35:11.339267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:35:11.339267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:35:11.339267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:35:11.339267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:11.339267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 368/758] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jun 27 15:35:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:11.339267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadc62e-af66-4300-bdba-e6b44ded5758 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:11.343673 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c4570a-6644-4d2f-9760-2aeb2cd97fb0 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:11.355057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:11.393705 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc24b62-5be9-415d-9564-fcb4004a044d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:11.394503 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc24b62-5be9-415d-9564-fcb4004a044d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:11.394755 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc24b62-5be9-415d-9564-fcb4004a044d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:11.395027 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc24b62-5be9-415d-9564-fcb4004a044d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:11.404852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc24b62-5be9-415d-9564-fcb4004a044d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:11.406334 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc24b62-5be9-415d-9564-fcb4004a044d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:11.426067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-3dc24b62-5be9-415d-9564-fcb4004a044d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:11.426067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:11.426067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:11.426067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:11.426067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:11.426067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:11.426067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:11.426067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:11.426067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:11.426067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:11.426067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:11.426067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:11.426067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:11.426067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:11.426067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:11.427049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:11.427049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:11.427049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:11.427049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:11.427049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:11.427049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:11.427049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:11.427049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:11.427049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:11.427049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:11.427049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:11.427049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:11.427049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:11.427049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:11.427049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:11.427049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:11.427049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:11.427923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:11.427923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:11.427923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:11.427923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:11.427923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:11.427923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:11.427923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:11.427923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:11.428371 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 391/759] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:11.429000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc24b62-5be9-415d-9564-fcb4004a044d None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:11.471687 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadc62e-af66-4300-bdba-e6b44ded5758 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:11.513539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da217564-475d-4c10-bc83-e1d6b3f9d7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:11.514267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da217564-475d-4c10-bc83-e1d6b3f9d7fc None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:11.514589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da217564-475d-4c10-bc83-e1d6b3f9d7fc None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:11.514905 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da217564-475d-4c10-bc83-e1d6b3f9d7fc None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:11.515715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da217564-475d-4c10-bc83-e1d6b3f9d7fc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:11.516181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da217564-475d-4c10-bc83-e1d6b3f9d7fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:11.517739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da217564-475d-4c10-bc83-e1d6b3f9d7fc None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:11.523509 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 369/760] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Jun 27 15:35:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:11.523868 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da217564-475d-4c10-bc83-e1d6b3f9d7fc None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:11.551308 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:11.577111 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc24b62-5be9-415d-9564-fcb4004a044d None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:11.613307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:11.614930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342a928-80a9-4471-a223-443738d8cbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:11.615573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342a928-80a9-4471-a223-443738d8cbec None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:11.615877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342a928-80a9-4471-a223-443738d8cbec None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:11.616143 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342a928-80a9-4471-a223-443738d8cbec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:11.616972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342a928-80a9-4471-a223-443738d8cbec None admin] RBAC: Adding query filter params (role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.project.id=7be83fb6f20341139147373837320148, user.id=8e83ea69ded040448fe75caa221ad377) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:11.617282 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342a928-80a9-4471-a223-443738d8cbec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:11.621125 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342a928-80a9-4471-a223-443738d8cbec None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:11.625383 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 392/761] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:35:11 2022] GET /identity/v3/role_assignments?user.id=8e83ea69ded040448fe75caa221ad377&scope.project.id=7be83fb6f20341139147373837320148&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:11.626224 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342a928-80a9-4471-a223-443738d8cbec None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:11.659663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da217564-475d-4c10-bc83-e1d6b3f9d7fc None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:11.705732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d17da5-eaff-42c1-9c7b-509870d7d7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:11.706422 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d17da5-eaff-42c1-9c7b-509870d7d7c0 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:11.706709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d17da5-eaff-42c1-9c7b-509870d7d7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:11.706987 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d17da5-eaff-42c1-9c7b-509870d7d7c0 None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:11.707947 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d17da5-eaff-42c1-9c7b-509870d7d7c0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:11.708406 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d17da5-eaff-42c1-9c7b-509870d7d7c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:11.710074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d17da5-eaff-42c1-9c7b-509870d7d7c0 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:11.733203 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 370/762] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Jun 27 15:35:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:11.734278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d17da5-eaff-42c1-9c7b-509870d7d7c0 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:11.767828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342a928-80a9-4471-a223-443738d8cbec None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:11.817969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ee011-a8ea-400b-bd01-da78d571d696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:11.818639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ee011-a8ea-400b-bd01-da78d571d696 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:11.818932 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ee011-a8ea-400b-bd01-da78d571d696 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:11.819220 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ee011-a8ea-400b-bd01-da78d571d696 None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:11.820011 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ee011-a8ea-400b-bd01-da78d571d696 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:11.820328 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ee011-a8ea-400b-bd01-da78d571d696 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:11.822636 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ee011-a8ea-400b-bd01-da78d571d696 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:11.836353 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 393/763] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:11.841617 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067aa5c3-0b1e-4a06-8460-97258bd6988e None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:11.842069 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067aa5c3-0b1e-4a06-8460-97258bd6988e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:11.842360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067aa5c3-0b1e-4a06-8460-97258bd6988e None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:11.843899 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 394/764] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:11.844481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067aa5c3-0b1e-4a06-8460-97258bd6988e None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:11.859013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:11.877010 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d17da5-eaff-42c1-9c7b-509870d7d7c0 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:11.921000 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:11.927744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31038f5a-3978-4933-9d3f-6ea010b203b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:11.928421 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31038f5a-3978-4933-9d3f-6ea010b203b4 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:11.928703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31038f5a-3978-4933-9d3f-6ea010b203b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:11.928978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31038f5a-3978-4933-9d3f-6ea010b203b4 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:11.929747 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31038f5a-3978-4933-9d3f-6ea010b203b4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:11.930061 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31038f5a-3978-4933-9d3f-6ea010b203b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:11.932243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31038f5a-3978-4933-9d3f-6ea010b203b4 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:11.946023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 371/765] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Jun 27 15:35:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:11.946023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31038f5a-3978-4933-9d3f-6ea010b203b4 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:11.991279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067aa5c3-0b1e-4a06-8460-97258bd6988e None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:12.035791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ca3bb-c635-44ae-adc1-eed3dc8503ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:12.036451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ca3bb-c635-44ae-adc1-eed3dc8503ec None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:12.036738 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ca3bb-c635-44ae-adc1-eed3dc8503ec None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:12.037019 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ca3bb-c635-44ae-adc1-eed3dc8503ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:12.085194 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31038f5a-3978-4933-9d3f-6ea010b203b4 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:12.120065 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ca3bb-c635-44ae-adc1-eed3dc8503ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:12.121542 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ca3bb-c635-44ae-adc1-eed3dc8503ec None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:12.126944 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83fe88f-569b-44ed-96ac-7c647b3b1e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:12.127909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fe88f-569b-44ed-96ac-7c647b3b1e1a None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:12.128136 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fe88f-569b-44ed-96ac-7c647b3b1e1a None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:12.128413 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fe88f-569b-44ed-96ac-7c647b3b1e1a None admin] PATH_INFO: `/v3/projects/7be83fb6f20341139147373837320148/users/1c8f6262cf24419daf803ae71aed24f5/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:12.130850 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fe88f-569b-44ed-96ac-7c647b3b1e1a None admin] RBAC: Authorizing `identity:create_grant(project_id=7be83fb6f20341139147373837320148, user_id=1c8f6262cf24419daf803ae71aed24f5, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:12.132701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fe88f-569b-44ed-96ac-7c647b3b1e1a None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:12.133293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:12.149226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 372/766] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:35:11 2022] PUT /identity/v3/projects/7be83fb6f20341139147373837320148/users/1c8f6262cf24419daf803ae71aed24f5/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:12.151185 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cff0d-ac79-482f-a7d8-dfa9873d0759 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:12.151680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cff0d-ac79-482f-a7d8-dfa9873d0759 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:12.152153 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cff0d-ac79-482f-a7d8-dfa9873d0759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:12.174226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-108cff0d-ac79-482f-a7d8-dfa9873d0759 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:12.175919 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-108cff0d-ac79-482f-a7d8-dfa9873d0759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:12.237968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:12.266322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 373/767] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:12.267538 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108cff0d-ac79-482f-a7d8-dfa9873d0759 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:12.281229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 395/768] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:35:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:35:12.287007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ca3bb-c635-44ae-adc1-eed3dc8503ec None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:12.421751 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108cff0d-ac79-482f-a7d8-dfa9873d0759 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:12.440267 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ca3bb-c635-44ae-adc1-eed3dc8503ec None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:12.459143 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:12.469434 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053d849-a767-4a7b-a365-3a6c1723effa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:12.470288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053d849-a767-4a7b-a365-3a6c1723effa None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:12.470573 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053d849-a767-4a7b-a365-3a6c1723effa None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:12.470847 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053d849-a767-4a7b-a365-3a6c1723effa None admin] PATH_INFO: `/v3/endpoints` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:12.471662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053d849-a767-4a7b-a365-3a6c1723effa None admin] RBAC: Adding query filter params (interface=public, service_id=534c1cdc612d4a07a9d1e9ddb8fa35c6, region_id=RegionOne) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:12.471974 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053d849-a767-4a7b-a365-3a6c1723effa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:12.474076 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6053d849-a767-4a7b-a365-3a6c1723effa None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:12.489735 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 374/769] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Jun 27 15:35:12 2022] GET /identity/v3/endpoints?service_id=534c1cdc612d4a07a9d1e9ddb8fa35c6&interface=public®ion_id=RegionOne => generated 194 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:12.490834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053d849-a767-4a7b-a365-3a6c1723effa None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:12.494399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c3ae9-230e-4c77-a384-55686e3c5fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:12.495555 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c3ae9-230e-4c77-a384-55686e3c5fd5 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:12.495838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c3ae9-230e-4c77-a384-55686e3c5fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:12.497133 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c3ae9-230e-4c77-a384-55686e3c5fd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:12.511966 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c3ae9-230e-4c77-a384-55686e3c5fd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:12.514750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822c3ae9-230e-4c77-a384-55686e3c5fd5 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:12.543363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-822c3ae9-230e-4c77-a384-55686e3c5fd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:12.543363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:12.543363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:12.543363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:12.543363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:12.543363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:12.543363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:12.543363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:12.543363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:12.543363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:12.543363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:12.543363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:12.543363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:12.543363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:12.543363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:12.544305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:12.544305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:12.544305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:12.544305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:12.544305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:12.544305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:12.544305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:12.544305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:12.544305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:12.544305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:12.544305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:12.544305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:12.544305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:12.544305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:12.544305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:12.544305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:12.544305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:12.545155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:12.545155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:12.545155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:12.545155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:12.545377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 396/770] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:12.549682 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c3ae9-230e-4c77-a384-55686e3c5fd5 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:12.563432 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:12.627446 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053d849-a767-4a7b-a365-3a6c1723effa None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:12.667881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad5e7d-5274-4387-b556-e578a5345b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:12.668810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad5e7d-5274-4387-b556-e578a5345b0c None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:12.669093 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad5e7d-5274-4387-b556-e578a5345b0c None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:12.669369 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad5e7d-5274-4387-b556-e578a5345b0c None admin] PATH_INFO: `/v3/services/network` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:12.670075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad5e7d-5274-4387-b556-e578a5345b0c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:12.671720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad5e7d-5274-4387-b556-e578a5345b0c None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:12.684609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c3ae9-230e-4c77-a384-55686e3c5fd5 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:12.684796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-b4ad5e7d-5274-4387-b556-e578a5345b0c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 15:35:12.684796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:12.684796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:12.684796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:12.684796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:12.684796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:12.684796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:12.684796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:12.684796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 15:35:12.684796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:12.684796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:12.684796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:12.684796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:12.684796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:12.684796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:12.684796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:12.684796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:12.685720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:12.685720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:12.685720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:12.685720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:12.685720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:12.685720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:12.685720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:12.685720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:12.685720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:12.685720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:12.685720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:12.685720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:12.685720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:12.685720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:12.685720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:12.685720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:12.686562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:12.686562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:12.686562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:12.686562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:12.686562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:12.686562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:12.686562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:12.686562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:12.686562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:12.686562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:12.686562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:12.686562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:12.686562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 15:35:12.686562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:12.687304 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 375/771] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:12 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:12.688213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad5e7d-5274-4387-b556-e578a5345b0c None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:12.736717 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d7bb6-47af-4198-b4f7-560fc8bb5fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:12.737409 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d7bb6-47af-4198-b4f7-560fc8bb5fad None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:12.737723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d7bb6-47af-4198-b4f7-560fc8bb5fad None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:12.738004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d7bb6-47af-4198-b4f7-560fc8bb5fad None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:12.738707 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d7bb6-47af-4198-b4f7-560fc8bb5fad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:12.740792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d7bb6-47af-4198-b4f7-560fc8bb5fad None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:12.753770 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 397/772] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Jun 27 15:35:12 2022] GET /identity/v3/services => generated 772 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:12.754839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d7bb6-47af-4198-b4f7-560fc8bb5fad None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:12.840699 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad5e7d-5274-4387-b556-e578a5345b0c None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:12.843692 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:12.858160 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:12.885253 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81f241-9d32-48ce-a868-784b54879905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:12.885986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81f241-9d32-48ce-a868-784b54879905 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:12.888408 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81f241-9d32-48ce-a868-784b54879905 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:12.888739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81f241-9d32-48ce-a868-784b54879905 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:12.889522 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81f241-9d32-48ce-a868-784b54879905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:12.889866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81f241-9d32-48ce-a868-784b54879905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:12.891322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81f241-9d32-48ce-a868-784b54879905 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:12.922592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 376/773] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:12.932692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81f241-9d32-48ce-a868-784b54879905 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:12.987375 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d7bb6-47af-4198-b4f7-560fc8bb5fad None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:13.041840 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8afe2f1-9c90-44b7-9cb1-3cddcc9f1b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:13.042716 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afe2f1-9c90-44b7-9cb1-3cddcc9f1b28 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:13.043007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afe2f1-9c90-44b7-9cb1-3cddcc9f1b28 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:13.043565 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afe2f1-9c90-44b7-9cb1-3cddcc9f1b28 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:13.044797 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8afe2f1-9c90-44b7-9cb1-3cddcc9f1b28 None admin] RBAC: Adding query filter params (name=network) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:13.045175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8afe2f1-9c90-44b7-9cb1-3cddcc9f1b28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:13.046896 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8afe2f1-9c90-44b7-9cb1-3cddcc9f1b28 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:13.053761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 398/774] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jun 27 15:35:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:13.061177 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8afe2f1-9c90-44b7-9cb1-3cddcc9f1b28 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:13.114888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81f241-9d32-48ce-a868-784b54879905 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:13.162458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952d542-5ebe-48ac-a372-26bf66bd2929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:13.163270 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952d542-5ebe-48ac-a372-26bf66bd2929 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:13.163552 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952d542-5ebe-48ac-a372-26bf66bd2929 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:13.163826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952d542-5ebe-48ac-a372-26bf66bd2929 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:13.183300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952d542-5ebe-48ac-a372-26bf66bd2929 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:13.185111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952d542-5ebe-48ac-a372-26bf66bd2929 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:13.215288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-6952d542-5ebe-48ac-a372-26bf66bd2929 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:13.215288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:13.215288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:13.215288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:13.215288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:13.215288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:13.215288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:13.215288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:13.215288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:13.215288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:13.215288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:13.215288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:13.215288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:13.215288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:13.215288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:13.216288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:13.216288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:13.216288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:13.216288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:13.216288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:13.216288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:13.216288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:13.216288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:13.216288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:13.216288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:13.216288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:13.216288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:13.216288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:13.216288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:13.216288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:13.216288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:13.216288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:13.217173 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:13.217173 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:13.217173 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:13.217173 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:13.217173 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:13.217173 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:13.217173 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:13.217173 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:13.229663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 377/775] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:13.245279 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952d542-5ebe-48ac-a372-26bf66bd2929 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:13.338068 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:13.345575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:13.361847 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8afe2f1-9c90-44b7-9cb1-3cddcc9f1b28 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:13.427844 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274735ab-9913-44b3-9dd2-63f5ae359e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:13.428532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274735ab-9913-44b3-9dd2-63f5ae359e6b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:13.428817 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274735ab-9913-44b3-9dd2-63f5ae359e6b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:13.429094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274735ab-9913-44b3-9dd2-63f5ae359e6b None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:13.429886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274735ab-9913-44b3-9dd2-63f5ae359e6b None admin] RBAC: Adding query filter params (type=network) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:13.430210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274735ab-9913-44b3-9dd2-63f5ae359e6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:13.440653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274735ab-9913-44b3-9dd2-63f5ae359e6b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:13.453759 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 399/776] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jun 27 15:35:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:13.472461 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274735ab-9913-44b3-9dd2-63f5ae359e6b None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:13.502970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952d542-5ebe-48ac-a372-26bf66bd2929 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:13.599997 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274735ab-9913-44b3-9dd2-63f5ae359e6b None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:13.623375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833aa2a-e7e7-4e3a-bf45-74d7f2cf6649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:13.624382 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833aa2a-e7e7-4e3a-bf45-74d7f2cf6649 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:13.624845 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833aa2a-e7e7-4e3a-bf45-74d7f2cf6649 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:13.625285 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833aa2a-e7e7-4e3a-bf45-74d7f2cf6649 None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:13.626560 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833aa2a-e7e7-4e3a-bf45-74d7f2cf6649 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:13.626878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833aa2a-e7e7-4e3a-bf45-74d7f2cf6649 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:13.628410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833aa2a-e7e7-4e3a-bf45-74d7f2cf6649 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:13.637324 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0b1f1-dbb6-4b99-874f-824c9717dc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:13.638027 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0b1f1-dbb6-4b99-874f-824c9717dc34 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:13.638319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0b1f1-dbb6-4b99-874f-824c9717dc34 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:13.638606 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0b1f1-dbb6-4b99-874f-824c9717dc34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:13.639413 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0b1f1-dbb6-4b99-874f-824c9717dc34 None admin] RBAC: Adding query filter params (interface=public, service_id=0ce3c36eb1194f09804b0c3c25bae251, region_id=RegionOne) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:13.639732 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0b1f1-dbb6-4b99-874f-824c9717dc34 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:13.641260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0b1f1-dbb6-4b99-874f-824c9717dc34 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:13.656081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 400/777] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Jun 27 15:35:13 2022] GET /identity/v3/endpoints?service_id=0ce3c36eb1194f09804b0c3c25bae251&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:13.660813 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0b1f1-dbb6-4b99-874f-824c9717dc34 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:13.668082 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 378/778] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:13.686717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833aa2a-e7e7-4e3a-bf45-74d7f2cf6649 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:13.688558 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:13.785666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0b1f1-dbb6-4b99-874f-824c9717dc34 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:13.829169 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6f5ce-c846-4f7c-a784-ab6f113c7f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:13.829878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6f5ce-c846-4f7c-a784-ab6f113c7f2c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:13.830167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6f5ce-c846-4f7c-a784-ab6f113c7f2c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:13.830447 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6f5ce-c846-4f7c-a784-ab6f113c7f2c None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:13.831200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6f5ce-c846-4f7c-a784-ab6f113c7f2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:13.832577 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6f5ce-c846-4f7c-a784-ab6f113c7f2c None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:13.845792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 401/779] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Jun 27 15:35:13 2022] GET /identity/v3/services => generated 772 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:13.847355 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303e46e-c11e-457f-9c41-7a18d591f51d None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:13.847648 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303e46e-c11e-457f-9c41-7a18d591f51d None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:13.847933 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303e46e-c11e-457f-9c41-7a18d591f51d None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:13.849300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 402/780] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:13.855444 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f542a1-363a-4277-9532-a1b3aaca2d12 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:13.855775 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f542a1-363a-4277-9532-a1b3aaca2d12 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:13.856003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f542a1-363a-4277-9532-a1b3aaca2d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:13.881903 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-33f542a1-363a-4277-9532-a1b3aaca2d12 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:13.883139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-33f542a1-363a-4277-9532-a1b3aaca2d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:13.888030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:13.899948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833aa2a-e7e7-4e3a-bf45-74d7f2cf6649 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:13.946493 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141caed-77e8-4e1a-b05d-1af2e5bee83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:13.947631 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141caed-77e8-4e1a-b05d-1af2e5bee83e None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:13.948058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141caed-77e8-4e1a-b05d-1af2e5bee83e None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:13.948498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141caed-77e8-4e1a-b05d-1af2e5bee83e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:13.950086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7141caed-77e8-4e1a-b05d-1af2e5bee83e None admin] RBAC: Adding query filter params (role.id=805851682adf4debbe99b1681c9fcdfc, scope.project.id=7be83fb6f20341139147373837320148, user.id=6655366e7eb84a26aa3f7fbe88af62b5) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:13.950617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7141caed-77e8-4e1a-b05d-1af2e5bee83e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:13.952340 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7141caed-77e8-4e1a-b05d-1af2e5bee83e None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:13.973932 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 379/781] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:35:13 2022] GET /identity/v3/role_assignments?user.id=6655366e7eb84a26aa3f7fbe88af62b5&scope.project.id=7be83fb6f20341139147373837320148&role.id=805851682adf4debbe99b1681c9fcdfc => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:14.019123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 403/782] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:35:14.055144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141caed-77e8-4e1a-b05d-1af2e5bee83e None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:14.185823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141caed-77e8-4e1a-b05d-1af2e5bee83e None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:14.231000 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421442e0-de6f-4d7d-b97b-fa120d2cf9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:14.231815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421442e0-de6f-4d7d-b97b-fa120d2cf9a5 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:14.233374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421442e0-de6f-4d7d-b97b-fa120d2cf9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:14.233869 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421442e0-de6f-4d7d-b97b-fa120d2cf9a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:14.249594 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421442e0-de6f-4d7d-b97b-fa120d2cf9a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:14.257956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-421442e0-de6f-4d7d-b97b-fa120d2cf9a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:35:14.257956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:14.257956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:14.257956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:14.257956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:14.257956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:14.257956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:14.257956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:14.257956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:14.257956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:14.257956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:14.257956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:14.257956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:14.257956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:14.257956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:14.257956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:14.260675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:14.260675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:14.260675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:14.260675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:14.260675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:14.260675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:14.260675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:14.260675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:14.260675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:14.260675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:14.260675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:14.260675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:14.260675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:14.260675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:14.260675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:14.260675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:14.262665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:14.262665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:14.262665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:14.262665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:14.262665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:14.262665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:14.262665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:14.262665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:14.262665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:14.262665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:14.262665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:35:14.262665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:14.262665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 380/783] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:35:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:14.276360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f542a1-363a-4277-9532-a1b3aaca2d12 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:14.302253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:14.437451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f542a1-363a-4277-9532-a1b3aaca2d12 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:14.488206 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d3a2e-4a5f-40b6-8b3d-00b3f99d5b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:14.488997 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d3a2e-4a5f-40b6-8b3d-00b3f99d5b27 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:14.489187 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d3a2e-4a5f-40b6-8b3d-00b3f99d5b27 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:14.489398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d3a2e-4a5f-40b6-8b3d-00b3f99d5b27 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:14.490172 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d3a2e-4a5f-40b6-8b3d-00b3f99d5b27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:14.490470 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d3a2e-4a5f-40b6-8b3d-00b3f99d5b27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:14.491830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d3a2e-4a5f-40b6-8b3d-00b3f99d5b27 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:14.505450 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 404/784] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:35:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:14.512043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421442e0-de6f-4d7d-b97b-fa120d2cf9a5 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:14.545592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:14.589837 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b431db0-b9e6-4a00-820b-172a9b8d92d5 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:14.590212 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b431db0-b9e6-4a00-820b-172a9b8d92d5 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:14.590585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b431db0-b9e6-4a00-820b-172a9b8d92d5 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:14.592307 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 405/785] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:14.597607 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156b4c4-7c12-429b-b558-590ce5e9ce44 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:14.598000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156b4c4-7c12-429b-b558-590ce5e9ce44 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:14.598347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156b4c4-7c12-429b-b558-590ce5e9ce44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:14.617030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-b156b4c4-7c12-429b-b558-590ce5e9ce44 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:14.619176 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-b156b4c4-7c12-429b-b558-590ce5e9ce44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:14.695712 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421442e0-de6f-4d7d-b97b-fa120d2cf9a5 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:14.741831 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 406/786] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:35:14.762893 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be7420-6f5e-4686-a6b3-0f42274c39da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:14.763766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be7420-6f5e-4686-a6b3-0f42274c39da None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:14.764055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be7420-6f5e-4686-a6b3-0f42274c39da None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:14.764335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be7420-6f5e-4686-a6b3-0f42274c39da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:14.782365 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be7420-6f5e-4686-a6b3-0f42274c39da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:14.783969 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be7420-6f5e-4686-a6b3-0f42274c39da None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:14.797688 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156b4c4-7c12-429b-b558-590ce5e9ce44 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:14.803323 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:14.813177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-e8be7420-6f5e-4686-a6b3-0f42274c39da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:14.813177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:14.813177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:14.813177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:14.813177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:14.813177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:14.813177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:14.813177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:14.813177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:14.813177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:14.813177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:14.813177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:14.813177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:14.813177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:14.813177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:14.814351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:14.814351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:14.814351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:14.814351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:14.814351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:14.814351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:14.814351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:14.814351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:14.814351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:14.814351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:14.814351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:14.814351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:14.814351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:14.814351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:14.814351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:14.814351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:14.814351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:14.815481 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:14.815481 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:14.815481 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:14.815481 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:14.817573 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 381/787] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:14.825066 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be7420-6f5e-4686-a6b3-0f42274c39da None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:14.942837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:15.016934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be7420-6f5e-4686-a6b3-0f42274c39da None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:15.020727 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156b4c4-7c12-429b-b558-590ce5e9ce44 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:15.068738 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d53dfe-f4f3-42ec-97bf-03d9130246b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:15.069428 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d53dfe-f4f3-42ec-97bf-03d9130246b5 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.069741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d53dfe-f4f3-42ec-97bf-03d9130246b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.070026 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d53dfe-f4f3-42ec-97bf-03d9130246b5 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.070769 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d53dfe-f4f3-42ec-97bf-03d9130246b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:15.071115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d53dfe-f4f3-42ec-97bf-03d9130246b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:15.072533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d53dfe-f4f3-42ec-97bf-03d9130246b5 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:15.103689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ded57e-42d5-4358-9519-afe2b702bb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:15.104506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ded57e-42d5-4358-9519-afe2b702bb48 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.104791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ded57e-42d5-4358-9519-afe2b702bb48 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.105068 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ded57e-42d5-4358-9519-afe2b702bb48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.105592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 382/788] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:15.108329 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d53dfe-f4f3-42ec-97bf-03d9130246b5 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:15.130717 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ded57e-42d5-4358-9519-afe2b702bb48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:15.138511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-d3ded57e-42d5-4358-9519-afe2b702bb48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:35:15.138511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:15.138511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:15.138511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:15.138511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:15.138511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:15.138511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:15.138511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:15.138511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:15.138511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:15.138511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:15.138511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:15.138511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:15.138511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:15.138511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:15.138511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:15.139561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:15.139561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:15.139561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:15.139561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:15.139561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:15.139561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:15.139561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:15.139561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:15.139561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:15.139561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:15.139561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:15.139561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:15.139561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:15.139561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:15.139561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:15.139561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:15.140395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:15.140395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:15.140395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:15.140395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:15.140395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:15.140395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:15.140395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:15.140395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:15.140395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:15.140395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:15.140395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:35:15.140395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:15.140395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 407/789] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:35:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:15.153513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ded57e-42d5-4358-9519-afe2b702bb48 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:15.283649 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d53dfe-f4f3-42ec-97bf-03d9130246b5 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:15.284201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:15.314815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:15.326782 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ded57e-42d5-4358-9519-afe2b702bb48 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:15.327140 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13ad4e-2f88-466c-a3f2-e74362d59981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:15.328200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13ad4e-2f88-466c-a3f2-e74362d59981 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.333660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13ad4e-2f88-466c-a3f2-e74362d59981 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.333660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13ad4e-2f88-466c-a3f2-e74362d59981 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.346658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b13ad4e-2f88-466c-a3f2-e74362d59981 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:15.348142 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b13ad4e-2f88-466c-a3f2-e74362d59981 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:15.374586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-8b13ad4e-2f88-466c-a3f2-e74362d59981 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:35:15.374586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:15.374586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:15.374586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:15.374586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:15.374586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:15.374586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:15.374586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:15.374586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:15.374586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:15.374586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:15.374586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:15.374586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:35:15.374586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:35:15.374586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:35:15.374586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:35:15.375674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:15.375674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:15.375674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:15.375674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:15.375674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:15.375674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:15.375674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:15.375674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:15.375674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:15.375674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:15.375674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:15.375674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:15.375674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:15.375674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:15.375674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:15.375674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:15.375674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:15.375674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:15.377235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:15.377235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:15.377235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:35:15.377235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:35:15.377235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:35:15.377235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:35:15.377235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:35:15.377235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:35:15.377235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:35:15.377235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:15.377235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 383/790] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:35:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:15.390420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13ad4e-2f88-466c-a3f2-e74362d59981 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:15.401691 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b999537-f896-438d-8dcc-4263f283d76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:15.402382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b999537-f896-438d-8dcc-4263f283d76e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.402669 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b999537-f896-438d-8dcc-4263f283d76e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.402952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b999537-f896-438d-8dcc-4263f283d76e None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.403727 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b999537-f896-438d-8dcc-4263f283d76e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:15.404039 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b999537-f896-438d-8dcc-4263f283d76e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:15.405425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b999537-f896-438d-8dcc-4263f283d76e None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:15.420139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 408/791] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:35:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:15.425025 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b999537-f896-438d-8dcc-4263f283d76e None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:15.535476 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13ad4e-2f88-466c-a3f2-e74362d59981 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:15.553531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b999537-f896-438d-8dcc-4263f283d76e None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:15.582301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:15.592879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71028002-af4f-47ef-a926-e26b1b101fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:15.594053 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71028002-af4f-47ef-a926-e26b1b101fb6 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.594165 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea8470-c40a-44b5-ae0f-6e72c94abd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:15.594710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71028002-af4f-47ef-a926-e26b1b101fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.595120 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea8470-c40a-44b5-ae0f-6e72c94abd32 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.595409 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea8470-c40a-44b5-ae0f-6e72c94abd32 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.595511 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71028002-af4f-47ef-a926-e26b1b101fb6 None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.595729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea8470-c40a-44b5-ae0f-6e72c94abd32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.596960 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71028002-af4f-47ef-a926-e26b1b101fb6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:15.597476 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71028002-af4f-47ef-a926-e26b1b101fb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:15.599932 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71028002-af4f-47ef-a926-e26b1b101fb6 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:15.609266 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea8470-c40a-44b5-ae0f-6e72c94abd32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:15.610762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea8470-c40a-44b5-ae0f-6e72c94abd32 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:15.623661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 384/792] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Jun 27 15:35:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:15.624901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34062b92-04df-4f9f-936f-6c7f0b66b6ff None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.625192 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34062b92-04df-4f9f-936f-6c7f0b66b6ff None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.625472 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34062b92-04df-4f9f-936f-6c7f0b66b6ff None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.627331 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 385/793] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:15.628552 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34062b92-04df-4f9f-936f-6c7f0b66b6ff None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:15.639178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-fbea8470-c40a-44b5-ae0f-6e72c94abd32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:15.639178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:15.639178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:15.639178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:15.639178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:15.639178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:15.639178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:15.639178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:15.639178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:15.639178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:15.639178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:15.639178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:15.639178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:15.639178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:15.639178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:15.640099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:15.640099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:15.640099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:15.640099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:15.640099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:15.640099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:15.640099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:15.640099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:15.640099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:15.640099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:15.640099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:15.640099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:15.640099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:15.640099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:15.640099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:15.640099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:15.640099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:15.640985 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:15.640985 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:15.640985 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:15.640985 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:15.643971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 409/794] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:15.645537 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f649c-933c-4e9d-a888-61077661edc3 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.645992 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f649c-933c-4e9d-a888-61077661edc3 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.646407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f649c-933c-4e9d-a888-61077661edc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.666354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-051f649c-933c-4e9d-a888-61077661edc3 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:15.667691 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-051f649c-933c-4e9d-a888-61077661edc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:15.695832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:15.748601 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 410/795] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:15.749833 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f649c-933c-4e9d-a888-61077661edc3 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:15.779525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34062b92-04df-4f9f-936f-6c7f0b66b6ff None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:15.832653 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45304bf-0f32-4ef2-a001-9fe7df7b81b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:15.834498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45304bf-0f32-4ef2-a001-9fe7df7b81b1 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.835000 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45304bf-0f32-4ef2-a001-9fe7df7b81b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.835755 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45304bf-0f32-4ef2-a001-9fe7df7b81b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.851993 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45304bf-0f32-4ef2-a001-9fe7df7b81b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:15.854005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45304bf-0f32-4ef2-a001-9fe7df7b81b1 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:15.880431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-b45304bf-0f32-4ef2-a001-9fe7df7b81b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:15.880431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:15.880431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:15.880431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:15.880431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:15.880431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:15.880431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:15.880431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:15.880431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:15.880431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:15.880431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:15.880431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:15.880431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:15.880431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:15.880431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:15.881423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:15.881423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:15.881423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:15.881423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:15.881423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:15.881423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:15.881423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:15.881423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:15.881423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:15.881423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:15.881423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:15.881423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:15.881423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:15.881423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:15.881423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:15.881423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:15.881423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:15.882384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:15.882384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:15.882384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:15.882384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:15.888976 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 386/796] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:15.888976 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45304bf-0f32-4ef2-a001-9fe7df7b81b1 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:15.892096 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f649c-933c-4e9d-a888-61077661edc3 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:15.911044 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:15.945700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08224bd1-224a-428a-900f-2eac5be42f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:15.947697 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08224bd1-224a-428a-900f-2eac5be42f0c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:15.948493 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08224bd1-224a-428a-900f-2eac5be42f0c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:15.948986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08224bd1-224a-428a-900f-2eac5be42f0c None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:15.950097 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08224bd1-224a-428a-900f-2eac5be42f0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:15.950929 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08224bd1-224a-428a-900f-2eac5be42f0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:15.952987 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08224bd1-224a-428a-900f-2eac5be42f0c None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:15.973564 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 411/797] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:15.975710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08224bd1-224a-428a-900f-2eac5be42f0c None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:16.021833 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:16.027180 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45304bf-0f32-4ef2-a001-9fe7df7b81b1 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:16.067855 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941607ca-de20-4072-9f2c-f0acf865810c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:16.068894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941607ca-de20-4072-9f2c-f0acf865810c None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.069274 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941607ca-de20-4072-9f2c-f0acf865810c None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.069595 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941607ca-de20-4072-9f2c-f0acf865810c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.070384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941607ca-de20-4072-9f2c-f0acf865810c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:16.072826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941607ca-de20-4072-9f2c-f0acf865810c None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:16.086257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-941607ca-de20-4072-9f2c-f0acf865810c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 15:35:16.086257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:16.086257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:16.086257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:16.086257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:16.086257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:16.086257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:16.086257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:16.086257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 15:35:16.086257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:16.086257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:16.086257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:16.086257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:16.086257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:16.086257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:16.086257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:16.087874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:16.087874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:16.087874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:16.087874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:16.087874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:16.087874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:16.087874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:16.087874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:16.087874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:16.087874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:16.087874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:16.087874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:16.087874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:16.087874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:16.087874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:16.087874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:16.088724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:16.088724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:16.088724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:16.088724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:16.088724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:16.088724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:16.088724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:16.088724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:16.088724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:16.088724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:16.088724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:16.088724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:16.088724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:16.088724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 15:35:16.088724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:16.093926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 387/798] 10.222.0.20 () {56 vars in 1129 bytes} [Mon Jun 27 15:35:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:16.094838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941607ca-de20-4072-9f2c-f0acf865810c None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:16.133126 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08224bd1-224a-428a-900f-2eac5be42f0c None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:16.172521 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cafe33-f57a-402a-91ec-35fbc770a422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:16.173659 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cafe33-f57a-402a-91ec-35fbc770a422 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.174154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cafe33-f57a-402a-91ec-35fbc770a422 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.174628 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cafe33-f57a-402a-91ec-35fbc770a422 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.175615 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cafe33-f57a-402a-91ec-35fbc770a422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:16.176130 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cafe33-f57a-402a-91ec-35fbc770a422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:16.177759 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cafe33-f57a-402a-91ec-35fbc770a422 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:16.198802 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 412/799] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:16.200188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cafe33-f57a-402a-91ec-35fbc770a422 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:16.208472 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941607ca-de20-4072-9f2c-f0acf865810c None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:16.233253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:16.249278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f833d5-d8a3-4594-bd7d-3bdc6338be1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:16.250135 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f833d5-d8a3-4594-bd7d-3bdc6338be1b None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.250473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f833d5-d8a3-4594-bd7d-3bdc6338be1b None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.250753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f833d5-d8a3-4594-bd7d-3bdc6338be1b None admin] PATH_INFO: `/v3/users/nova` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.260576 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f833d5-d8a3-4594-bd7d-3bdc6338be1b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:16.262191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f833d5-d8a3-4594-bd7d-3bdc6338be1b None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:16.278977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-22f833d5-d8a3-4594-bd7d-3bdc6338be1b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:35:16.278977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:16.278977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:16.278977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:16.278977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:16.278977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:16.278977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:16.278977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:16.278977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:16.278977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:16.278977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:16.278977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:16.278977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:35:16.278977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:35:16.278977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:35:16.278977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:35:16.279996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:16.279996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:16.279996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:16.279996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:16.279996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:16.279996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:16.279996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:16.279996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:16.279996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:16.279996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:16.279996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:16.279996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:16.279996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:16.279996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:16.279996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:16.279996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:16.279996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:16.279996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:16.281306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:16.281306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:16.281306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:35:16.281306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:35:16.281306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:35:16.281306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:35:16.281306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:35:16.281306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:35:16.281306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:35:16.281306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:16.281306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 388/800] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Jun 27 15:35:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:16.281306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f833d5-d8a3-4594-bd7d-3bdc6338be1b None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:16.315387 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:16.322580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cafe33-f57a-402a-91ec-35fbc770a422 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:16.361222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbe2a8-e451-4acf-a956-2c8ce321f20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:16.362285 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbe2a8-e451-4acf-a956-2c8ce321f20a None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.362800 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbe2a8-e451-4acf-a956-2c8ce321f20a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.363277 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbe2a8-e451-4acf-a956-2c8ce321f20a None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.364193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbe2a8-e451-4acf-a956-2c8ce321f20a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:16.364706 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbe2a8-e451-4acf-a956-2c8ce321f20a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:16.366311 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbe2a8-e451-4acf-a956-2c8ce321f20a None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:16.372342 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 413/801] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Jun 27 15:35:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:16.373629 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbe2a8-e451-4acf-a956-2c8ce321f20a None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:16.412202 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f833d5-d8a3-4594-bd7d-3bdc6338be1b None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:16.450208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac3362-cec1-45f5-b8e5-94447d5675b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:16.451022 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac3362-cec1-45f5-b8e5-94447d5675b6 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.451311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac3362-cec1-45f5-b8e5-94447d5675b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.451595 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac3362-cec1-45f5-b8e5-94447d5675b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.459311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac3362-cec1-45f5-b8e5-94447d5675b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:16.460779 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac3362-cec1-45f5-b8e5-94447d5675b6 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:16.475117 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:16.476322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-caac3362-cec1-45f5-b8e5-94447d5675b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:16.476322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:16.476322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:16.476322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:16.476322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:16.476322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:16.476322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:16.476322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:16.476322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:16.476322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:16.476322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:16.476322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:16.476322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:16.476322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:16.476322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:16.478753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:16.478753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:16.478753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:16.478753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:16.478753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:16.478753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:16.478753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:16.478753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:16.478753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:16.478753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:16.478753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:16.478753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:16.478753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:16.478753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:16.478753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:16.478753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:16.478753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:16.480150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:16.480150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:16.480150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:16.480150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:16.480150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:16.480150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:16.480150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:16.480150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:16.480150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 389/802] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:16.480150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac3362-cec1-45f5-b8e5-94447d5675b6 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:16.489999 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbe2a8-e451-4acf-a956-2c8ce321f20a None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:16.525958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab14296-d289-41f1-87af-0b31a8cd256e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:16.526629 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab14296-d289-41f1-87af-0b31a8cd256e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.526913 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab14296-d289-41f1-87af-0b31a8cd256e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.527188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab14296-d289-41f1-87af-0b31a8cd256e None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.528026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab14296-d289-41f1-87af-0b31a8cd256e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:16.528330 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab14296-d289-41f1-87af-0b31a8cd256e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:16.529780 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab14296-d289-41f1-87af-0b31a8cd256e None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:16.550147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 414/803] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Jun 27 15:35:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:16.551479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3df6d-8ab6-4224-95b9-d146fefa6dd6 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.551767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3df6d-8ab6-4224-95b9-d146fefa6dd6 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.552042 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3df6d-8ab6-4224-95b9-d146fefa6dd6 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.553293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 415/804] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:16.554429 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72cfc4-d79b-4b8e-b676-e5d10d15e6a1 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.555098 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72cfc4-d79b-4b8e-b676-e5d10d15e6a1 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.555098 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72cfc4-d79b-4b8e-b676-e5d10d15e6a1 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.559514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 416/805] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:16.559733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72cfc4-d79b-4b8e-b676-e5d10d15e6a1 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:16.576785 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:16.597649 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac3362-cec1-45f5-b8e5-94447d5675b6 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:16.633420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a42e6f-1287-410f-96e8-a47b588d0c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:16.634829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a42e6f-1287-410f-96e8-a47b588d0c68 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.635187 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a42e6f-1287-410f-96e8-a47b588d0c68 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.635470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a42e6f-1287-410f-96e8-a47b588d0c68 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.636226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a42e6f-1287-410f-96e8-a47b588d0c68 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:16.636529 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a42e6f-1287-410f-96e8-a47b588d0c68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:16.637930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a42e6f-1287-410f-96e8-a47b588d0c68 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:16.647809 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 390/806] 10.222.0.20 () {56 vars in 1124 bytes} [Mon Jun 27 15:35:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:16.650301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a42e6f-1287-410f-96e8-a47b588d0c68 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:16.677997 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72cfc4-d79b-4b8e-b676-e5d10d15e6a1 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:16.713580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf58d4-90f0-459f-b5bf-ab80c690a2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:16.714260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf58d4-90f0-459f-b5bf-ab80c690a2b1 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.714540 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf58d4-90f0-459f-b5bf-ab80c690a2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.714814 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf58d4-90f0-459f-b5bf-ab80c690a2b1 None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.715595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf58d4-90f0-459f-b5bf-ab80c690a2b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:16.715903 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf58d4-90f0-459f-b5bf-ab80c690a2b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:16.717328 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf58d4-90f0-459f-b5bf-ab80c690a2b1 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:16.729309 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 417/807] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:16.730705 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667153e4-cd6d-4446-93f5-be5d75ced3c5 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.730987 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667153e4-cd6d-4446-93f5-be5d75ced3c5 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.731261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667153e4-cd6d-4446-93f5-be5d75ced3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.748247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-667153e4-cd6d-4446-93f5-be5d75ced3c5 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:16.749425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-667153e4-cd6d-4446-93f5-be5d75ced3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:16.752365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:16.762746 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a42e6f-1287-410f-96e8-a47b588d0c68 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:16.800831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410614a7-f266-4499-bda1-15ef97d7da5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:16.801496 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410614a7-f266-4499-bda1-15ef97d7da5d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.801818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410614a7-f266-4499-bda1-15ef97d7da5d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.802097 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410614a7-f266-4499-bda1-15ef97d7da5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.820054 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:16.824603 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 418/808] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:16.826201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57526e37-2ffe-4b67-9e8a-7d2574816731 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:16.826488 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57526e37-2ffe-4b67-9e8a-7d2574816731 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:16.826764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57526e37-2ffe-4b67-9e8a-7d2574816731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:16.845432 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-57526e37-2ffe-4b67-9e8a-7d2574816731 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:16.846674 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-57526e37-2ffe-4b67-9e8a-7d2574816731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:16.894027 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410614a7-f266-4499-bda1-15ef97d7da5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:16.895913 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410614a7-f266-4499-bda1-15ef97d7da5d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:16.921366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 419/809] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:16.922494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57526e37-2ffe-4b67-9e8a-7d2574816731 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:17.013248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 391/810] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:35:16 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:35:17.013937 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410614a7-f266-4499-bda1-15ef97d7da5d None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:17.014428 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:17.054345 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57526e37-2ffe-4b67-9e8a-7d2574816731 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:17.065189 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:17.098575 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa4004-2d58-4b91-9cff-48eadadb24ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:17.099239 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa4004-2d58-4b91-9cff-48eadadb24ed None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:17.099520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa4004-2d58-4b91-9cff-48eadadb24ed None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:17.099793 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa4004-2d58-4b91-9cff-48eadadb24ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:17.100540 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa4004-2d58-4b91-9cff-48eadadb24ed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:17.101933 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa4004-2d58-4b91-9cff-48eadadb24ed None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:17.123878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 420/811] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Jun 27 15:35:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:17.124715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa4004-2d58-4b91-9cff-48eadadb24ed None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:17.130570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410614a7-f266-4499-bda1-15ef97d7da5d None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:17.192323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce7b60-5a12-4a6b-b7ce-fa1a4a2c207a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:17.193160 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce7b60-5a12-4a6b-b7ce-fa1a4a2c207a None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:17.193453 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce7b60-5a12-4a6b-b7ce-fa1a4a2c207a None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:17.193762 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce7b60-5a12-4a6b-b7ce-fa1a4a2c207a None admin] PATH_INFO: `/v3/projects/7be83fb6f20341139147373837320148/users/8e83ea69ded040448fe75caa221ad377/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:17.197381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce7b60-5a12-4a6b-b7ce-fa1a4a2c207a None admin] RBAC: Authorizing `identity:create_grant(project_id=7be83fb6f20341139147373837320148, user_id=8e83ea69ded040448fe75caa221ad377, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:17.199361 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce7b60-5a12-4a6b-b7ce-fa1a4a2c207a None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:17.216586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 392/812] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:35:17 2022] PUT /identity/v3/projects/7be83fb6f20341139147373837320148/users/8e83ea69ded040448fe75caa221ad377/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:17.217545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce7b60-5a12-4a6b-b7ce-fa1a4a2c207a None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:17.276603 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa4004-2d58-4b91-9cff-48eadadb24ed None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:17.318400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:17.336544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd772a3-7156-4843-8146-c01cfb69ef7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:17.338232 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd772a3-7156-4843-8146-c01cfb69ef7a None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:17.339249 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd772a3-7156-4843-8146-c01cfb69ef7a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:17.339249 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd772a3-7156-4843-8146-c01cfb69ef7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:17.355324 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd772a3-7156-4843-8146-c01cfb69ef7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:17.364494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-8cd772a3-7156-4843-8146-c01cfb69ef7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:35:17.364494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:17.364494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:17.364494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:17.364494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:17.364494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:17.364494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:17.364494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:17.364494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:17.364494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:17.364494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:17.364494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:17.364494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:17.364494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:17.364494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:17.364494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:17.365589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:17.365589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:17.365589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:17.365589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:17.365589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:17.365589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:17.365589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:17.365589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:17.365589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:17.365589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:17.365589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:17.365589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:17.365589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:17.365589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:17.365589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:17.365589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:17.366442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:17.366442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:17.366442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:17.366442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:17.366442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:17.366442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:17.366442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:17.366442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:17.366442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:17.366442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:17.366442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:35:17.366442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:17.368138 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 421/813] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:35:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:17.373496 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd772a3-7156-4843-8146-c01cfb69ef7a None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:17.373709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce7b60-5a12-4a6b-b7ce-fa1a4a2c207a None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:17.373709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:17.425859 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf143c-611d-4cac-a90d-12d30f063d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:17.426788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf143c-611d-4cac-a90d-12d30f063d5f None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:17.427375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf143c-611d-4cac-a90d-12d30f063d5f None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:17.427375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf143c-611d-4cac-a90d-12d30f063d5f None admin] PATH_INFO: `/v3/services/network` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:17.428125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf143c-611d-4cac-a90d-12d30f063d5f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:17.429630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf143c-611d-4cac-a90d-12d30f063d5f None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:17.442732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-fabf143c-611d-4cac-a90d-12d30f063d5f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 15:35:17.442732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:17.442732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:17.442732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:17.442732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:17.442732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:17.442732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:17.442732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:17.442732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 15:35:17.442732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:17.442732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:17.442732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:17.442732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:17.442732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:17.442732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:17.442732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:17.442732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:17.443718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:17.443718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:17.443718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:17.443718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:17.443718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:17.443718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:17.443718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:17.443718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:17.443718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:17.443718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:17.443718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:17.443718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:17.443718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:17.443718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:17.443718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:17.443718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:17.444578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:17.444578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:17.444578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:17.444578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:17.444578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:17.444578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:17.444578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:17.444578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:17.444578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:17.444578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:17.444578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:17.444578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:17.444578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 15:35:17.444578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:17.449521 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 393/814] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:17 2022] GET /identity/v3/services/network => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:17.450558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf143c-611d-4cac-a90d-12d30f063d5f None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:17.523660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd772a3-7156-4843-8146-c01cfb69ef7a None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:17.564499 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2652f-48d6-4fe4-aa0a-97c1d9161b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:17.565403 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2652f-48d6-4fe4-aa0a-97c1d9161b4c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:17.565587 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2652f-48d6-4fe4-aa0a-97c1d9161b4c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:17.565866 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2652f-48d6-4fe4-aa0a-97c1d9161b4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:17.578986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2652f-48d6-4fe4-aa0a-97c1d9161b4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:17.580397 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2652f-48d6-4fe4-aa0a-97c1d9161b4c None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:17.594283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf143c-611d-4cac-a90d-12d30f063d5f None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:17.602889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-cbc2652f-48d6-4fe4-aa0a-97c1d9161b4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:17.602889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:17.602889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:17.602889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:17.602889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:17.602889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:17.602889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:17.602889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:17.602889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:17.602889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:17.602889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:17.602889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:17.602889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:17.602889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:17.602889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:17.603915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:17.603915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:17.603915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:17.603915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:17.603915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:17.603915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:17.603915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:17.603915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:17.603915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:17.603915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:17.603915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:17.603915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:17.603915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:17.603915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:17.603915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:17.603915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:17.603915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:17.604783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:17.604783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:17.604783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:17.604783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:17.607728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 422/815] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:17.608742 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2652f-48d6-4fe4-aa0a-97c1d9161b4c None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:17.638365 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ef983-59c1-414d-b917-be8d2ba6f452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:17.639208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ef983-59c1-414d-b917-be8d2ba6f452 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:17.639588 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ef983-59c1-414d-b917-be8d2ba6f452 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:17.639967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ef983-59c1-414d-b917-be8d2ba6f452 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:17.640814 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ef983-59c1-414d-b917-be8d2ba6f452 None admin] RBAC: Adding query filter params (name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:17.641221 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ef983-59c1-414d-b917-be8d2ba6f452 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:17.642801 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ef983-59c1-414d-b917-be8d2ba6f452 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:17.657297 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 394/816] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:35:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:17.658366 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ef983-59c1-414d-b917-be8d2ba6f452 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:17.663638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:17.693337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:17.744737 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2652f-48d6-4fe4-aa0a-97c1d9161b4c None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:17.788157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ef983-59c1-414d-b917-be8d2ba6f452 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:17.789545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40078b-087a-42f3-a671-a083af626989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:17.791131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40078b-087a-42f3-a671-a083af626989 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:17.792488 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40078b-087a-42f3-a671-a083af626989 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:17.792948 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40078b-087a-42f3-a671-a083af626989 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:17.793843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40078b-087a-42f3-a671-a083af626989 None admin] RBAC: Adding query filter params (name=network) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:17.794305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40078b-087a-42f3-a671-a083af626989 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:17.795853 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40078b-087a-42f3-a671-a083af626989 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:17.803083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 423/817] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jun 27 15:35:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:17.804361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40078b-087a-42f3-a671-a083af626989 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:17.840511 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e6f90-da5d-4e58-9e8a-a81df114c624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:17.841314 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e6f90-da5d-4e58-9e8a-a81df114c624 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:17.841606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e6f90-da5d-4e58-9e8a-a81df114c624 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:17.841929 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e6f90-da5d-4e58-9e8a-a81df114c624 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:17.842895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e6f90-da5d-4e58-9e8a-a81df114c624 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:17.843211 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e6f90-da5d-4e58-9e8a-a81df114c624 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:17.844731 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e6f90-da5d-4e58-9e8a-a81df114c624 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:17.860880 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 395/818] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:17.862051 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e6f90-da5d-4e58-9e8a-a81df114c624 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:17.931206 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40078b-087a-42f3-a671-a083af626989 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:17.933483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:17.969488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:17.975897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68dbd6b-9455-4dd4-9312-084aa4e37e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:17.977981 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dbd6b-9455-4dd4-9312-084aa4e37e49 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:17.978589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dbd6b-9455-4dd4-9312-084aa4e37e49 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:17.979094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dbd6b-9455-4dd4-9312-084aa4e37e49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:17.984478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e6f90-da5d-4e58-9e8a-a81df114c624 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:17.991805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68dbd6b-9455-4dd4-9312-084aa4e37e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:17.994368 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68dbd6b-9455-4dd4-9312-084aa4e37e49 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:18.015194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-e68dbd6b-9455-4dd4-9312-084aa4e37e49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:18.015194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:18.015194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:18.015194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:18.015194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:18.015194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:18.015194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:18.015194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:18.015194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:18.015194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:18.015194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:18.015194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:18.015194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:18.015194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:18.015194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:18.016131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:18.016131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:18.016131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:18.016131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:18.016131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:18.016131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:18.016131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:18.016131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:18.016131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:18.016131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:18.016131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:18.016131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:18.016131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:18.016131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:18.016131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:18.016131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:18.016131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:18.017022 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:18.017022 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:18.017022 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:18.017022 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:18.019626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 424/819] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:18.020760 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68dbd6b-9455-4dd4-9312-084aa4e37e49 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:18.024381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb06fc9-72ba-48f6-b472-084e473db903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:18.025150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb06fc9-72ba-48f6-b472-084e473db903 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:18.026291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb06fc9-72ba-48f6-b472-084e473db903 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:18.026291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb06fc9-72ba-48f6-b472-084e473db903 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:18.026600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb06fc9-72ba-48f6-b472-084e473db903 None admin] RBAC: Adding query filter params (type=network) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:18.026913 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb06fc9-72ba-48f6-b472-084e473db903 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:18.028354 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb06fc9-72ba-48f6-b472-084e473db903 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:18.034860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 396/820] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jun 27 15:35:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:18.035971 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb06fc9-72ba-48f6-b472-084e473db903 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:18.141717 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68dbd6b-9455-4dd4-9312-084aa4e37e49 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:18.162095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb06fc9-72ba-48f6-b472-084e473db903 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:18.179833 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a27310-b08a-461d-84d7-6b402c201d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:18.181809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a27310-b08a-461d-84d7-6b402c201d0d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:18.182325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a27310-b08a-461d-84d7-6b402c201d0d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:18.182817 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a27310-b08a-461d-84d7-6b402c201d0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:18.191298 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a27310-b08a-461d-84d7-6b402c201d0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:18.193260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a27310-b08a-461d-84d7-6b402c201d0d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:18.210652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-f4a27310-b08a-461d-84d7-6b402c201d0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:18.210652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:18.210652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:18.210652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:18.210652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:18.210652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:18.210652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:18.210652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:18.210652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:18.210652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:18.210652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:18.210652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:18.210652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:18.210652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:18.210652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:18.211643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:18.211643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:18.211643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:18.211643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:18.211643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:18.211643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:18.211643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:18.211643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:18.211643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:18.211643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:18.211643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:18.211643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:18.211643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:18.211643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:18.211643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:18.211643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:18.211643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:18.212529 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:18.212529 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:18.212529 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:18.212529 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:18.212529 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:18.212529 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:18.212529 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:18.212529 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:18.213413 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0bdb82-fa6b-4a71-a0c0-6608da9a07dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:18.213838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0bdb82-fa6b-4a71-a0c0-6608da9a07dc None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:18.214133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0bdb82-fa6b-4a71-a0c0-6608da9a07dc None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:18.214414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0bdb82-fa6b-4a71-a0c0-6608da9a07dc None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:18.215171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0bdb82-fa6b-4a71-a0c0-6608da9a07dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:18.215485 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0bdb82-fa6b-4a71-a0c0-6608da9a07dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:18.216435 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 425/821] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:18.216947 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0bdb82-fa6b-4a71-a0c0-6608da9a07dc None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:18.217811 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a27310-b08a-461d-84d7-6b402c201d0d None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:18.232884 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 397/822] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:18.233807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0bdb82-fa6b-4a71-a0c0-6608da9a07dc None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:18.236553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:18.253458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:18.337480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a27310-b08a-461d-84d7-6b402c201d0d None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:18.367259 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0bdb82-fa6b-4a71-a0c0-6608da9a07dc None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:18.377011 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fad11f-4f7b-40c6-920b-a60a14d46490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:18.377705 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fad11f-4f7b-40c6-920b-a60a14d46490 None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:18.377998 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fad11f-4f7b-40c6-920b-a60a14d46490 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:18.378280 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fad11f-4f7b-40c6-920b-a60a14d46490 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:18.379189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fad11f-4f7b-40c6-920b-a60a14d46490 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:18.380758 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fad11f-4f7b-40c6-920b-a60a14d46490 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:18.413666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 426/823] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Jun 27 15:35:18 2022] POST /identity/v3/endpoints => generated 327 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:18.413666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fad11f-4f7b-40c6-920b-a60a14d46490 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:18.441783 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139018a3-7d9a-4a18-9d18-d78ab5b116ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:18.443023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139018a3-7d9a-4a18-9d18-d78ab5b116ef None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:18.443703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139018a3-7d9a-4a18-9d18-d78ab5b116ef None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:18.444197 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139018a3-7d9a-4a18-9d18-d78ab5b116ef None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:18.445334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139018a3-7d9a-4a18-9d18-d78ab5b116ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:18.445920 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139018a3-7d9a-4a18-9d18-d78ab5b116ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:18.448938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139018a3-7d9a-4a18-9d18-d78ab5b116ef None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:18.482239 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 398/824] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:18.488617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139018a3-7d9a-4a18-9d18-d78ab5b116ef None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:18.495711 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:18.536304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fad11f-4f7b-40c6-920b-a60a14d46490 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:18.582430 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac0e44-fcf2-4e2f-a694-5e44fda06329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:18.583268 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac0e44-fcf2-4e2f-a694-5e44fda06329 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:18.583555 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac0e44-fcf2-4e2f-a694-5e44fda06329 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:18.583826 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac0e44-fcf2-4e2f-a694-5e44fda06329 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:18.594175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac0e44-fcf2-4e2f-a694-5e44fda06329 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:18.595679 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac0e44-fcf2-4e2f-a694-5e44fda06329 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:18.613830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-3aac0e44-fcf2-4e2f-a694-5e44fda06329 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:35:18.613830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:18.613830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:18.613830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:18.613830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:18.613830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:18.613830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:18.613830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:18.613830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:18.613830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:18.613830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:18.613830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:18.613830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:35:18.613830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:35:18.613830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:35:18.613830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:35:18.614835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:18.614835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:18.614835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:18.614835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:18.614835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:18.614835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:18.614835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:18.614835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:18.614835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:18.614835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:18.614835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:18.614835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:18.614835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:18.614835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:18.614835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:18.614835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:18.614835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:18.614835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:18.616051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:18.616051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:18.616051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:35:18.616051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:35:18.616051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:35:18.616051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:35:18.616051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:35:18.616051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:35:18.616051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:35:18.616051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:18.616051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 427/825] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jun 27 15:35:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:18.626772 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac0e44-fcf2-4e2f-a694-5e44fda06329 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:18.627985 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:18.661685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139018a3-7d9a-4a18-9d18-d78ab5b116ef None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:18.715077 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171bfe70-4fab-46b2-9254-5785296de377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:18.715757 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bfe70-4fab-46b2-9254-5785296de377 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:18.716049 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bfe70-4fab-46b2-9254-5785296de377 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:18.716328 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bfe70-4fab-46b2-9254-5785296de377 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:18.717175 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171bfe70-4fab-46b2-9254-5785296de377 None admin] RBAC: Adding query filter params (role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.project.id=7be83fb6f20341139147373837320148, user.id=1c8f6262cf24419daf803ae71aed24f5) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:18.717489 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171bfe70-4fab-46b2-9254-5785296de377 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:18.719565 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171bfe70-4fab-46b2-9254-5785296de377 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:18.727810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 399/826] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:35:18 2022] GET /identity/v3/role_assignments?user.id=1c8f6262cf24419daf803ae71aed24f5&scope.project.id=7be83fb6f20341139147373837320148&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:18.788026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac0e44-fcf2-4e2f-a694-5e44fda06329 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:18.824773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:18.841196 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcaaaa-e8dd-465e-982d-31074e1a3dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:18.841956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcaaaa-e8dd-465e-982d-31074e1a3dbe None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:18.842238 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcaaaa-e8dd-465e-982d-31074e1a3dbe None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:18.842513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcaaaa-e8dd-465e-982d-31074e1a3dbe None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:18.843387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affcaaaa-e8dd-465e-982d-31074e1a3dbe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:18.843696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affcaaaa-e8dd-465e-982d-31074e1a3dbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:18.845188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affcaaaa-e8dd-465e-982d-31074e1a3dbe None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:18.875417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 428/827] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Jun 27 15:35:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:18.882859 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171bfe70-4fab-46b2-9254-5785296de377 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:19.017351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171bfe70-4fab-46b2-9254-5785296de377 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:19.050612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:19.057588 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6d80a-4293-481f-9804-b7a75b76d7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:19.058453 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6d80a-4293-481f-9804-b7a75b76d7f7 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.058741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6d80a-4293-481f-9804-b7a75b76d7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.059023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6d80a-4293-481f-9804-b7a75b76d7f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.072501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6d80a-4293-481f-9804-b7a75b76d7f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:19.074125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6d80a-4293-481f-9804-b7a75b76d7f7 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:19.093766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-a9f6d80a-4293-481f-9804-b7a75b76d7f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:19.093766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:19.093766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:19.093766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:19.093766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:19.093766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:19.093766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:19.093766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:19.093766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:19.093766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:19.093766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:19.093766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:19.093766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:19.093766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:19.093766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:19.094706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:19.094706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:19.094706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:19.094706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:19.094706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:19.094706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:19.094706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:19.094706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:19.094706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:19.094706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:19.094706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:19.094706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:19.094706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:19.094706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:19.094706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:19.094706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:19.094706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:19.095796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:19.095796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:19.095796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:19.095796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:19.095796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 400/828] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:19.102379 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcaaaa-e8dd-465e-982d-31074e1a3dbe None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:19.227718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcaaaa-e8dd-465e-982d-31074e1a3dbe None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:19.266463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175fe77-e674-4892-b40a-d49f6db77a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:19.267126 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175fe77-e674-4892-b40a-d49f6db77a0b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.267413 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175fe77-e674-4892-b40a-d49f6db77a0b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.267692 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175fe77-e674-4892-b40a-d49f6db77a0b None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.268429 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175fe77-e674-4892-b40a-d49f6db77a0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:19.268743 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175fe77-e674-4892-b40a-d49f6db77a0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:19.270169 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175fe77-e674-4892-b40a-d49f6db77a0b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:19.283549 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 429/829] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:19.289428 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6d80a-4293-481f-9804-b7a75b76d7f7 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:19.400252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6d80a-4293-481f-9804-b7a75b76d7f7 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:19.436730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cc99e-4cdc-4fdd-a7fa-e7bac8337083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:19.437943 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cc99e-4cdc-4fdd-a7fa-e7bac8337083 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.438427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cc99e-4cdc-4fdd-a7fa-e7bac8337083 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.438896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cc99e-4cdc-4fdd-a7fa-e7bac8337083 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.446859 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cc99e-4cdc-4fdd-a7fa-e7bac8337083 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:19.448534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cc99e-4cdc-4fdd-a7fa-e7bac8337083 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:19.463823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-1b1cc99e-4cdc-4fdd-a7fa-e7bac8337083 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:19.463823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:19.463823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:19.463823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:19.463823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:19.463823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:19.463823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:19.463823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:19.463823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:19.463823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:19.463823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:19.463823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:19.463823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:19.463823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:19.463823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:19.464725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:19.464725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:19.464725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:19.464725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:19.464725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:19.464725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:19.464725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:19.464725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:19.464725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:19.464725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:19.464725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:19.464725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:19.464725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:19.464725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:19.464725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:19.464725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:19.464725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:19.465710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:19.465710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:19.465710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:19.465710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:19.465710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:19.465710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:19.465710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:19.465710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:19.472488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 401/830] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:19.474795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ca538-6631-479e-85d9-b179202152f3 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.475082 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ca538-6631-479e-85d9-b179202152f3 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.475359 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ca538-6631-479e-85d9-b179202152f3 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.476297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175fe77-e674-4892-b40a-d49f6db77a0b None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:19.478127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 402/831] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:19.483304 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbd1c8-1bdc-41ef-8cd1-57c6d2724785 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.483590 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbd1c8-1bdc-41ef-8cd1-57c6d2724785 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.483908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbd1c8-1bdc-41ef-8cd1-57c6d2724785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.502585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-59fbd1c8-1bdc-41ef-8cd1-57c6d2724785 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:19.503885 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-59fbd1c8-1bdc-41ef-8cd1-57c6d2724785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:19.536600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:19.551614 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:19.583363 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 403/832] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:35:19.585178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36537d4f-dea3-4a37-a15e-f31bb90bac77 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.585968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36537d4f-dea3-4a37-a15e-f31bb90bac77 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.586432 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36537d4f-dea3-4a37-a15e-f31bb90bac77 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.589075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 404/833] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:19.598857 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c50442-6aba-454d-878f-3c8f2aee5306 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.599007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175fe77-e674-4892-b40a-d49f6db77a0b None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:19.599435 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c50442-6aba-454d-878f-3c8f2aee5306 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.599435 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c50442-6aba-454d-878f-3c8f2aee5306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.624043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-17c50442-6aba-454d-878f-3c8f2aee5306 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:19.625231 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-17c50442-6aba-454d-878f-3c8f2aee5306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:19.635637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecfce7-c393-44da-9eb2-85fa9c7646bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:19.636298 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecfce7-c393-44da-9eb2-85fa9c7646bb None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.636790 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecfce7-c393-44da-9eb2-85fa9c7646bb None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.636899 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecfce7-c393-44da-9eb2-85fa9c7646bb None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.637610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ecfce7-c393-44da-9eb2-85fa9c7646bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:19.637945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ecfce7-c393-44da-9eb2-85fa9c7646bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:19.639425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ecfce7-c393-44da-9eb2-85fa9c7646bb None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:19.658742 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 430/834] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:19.659436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecfce7-c393-44da-9eb2-85fa9c7646bb None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:19.705769 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 405/835] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:19.707210 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c50442-6aba-454d-878f-3c8f2aee5306 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:19.772219 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecfce7-c393-44da-9eb2-85fa9c7646bb None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:19.783027 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:19.815853 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c507e4e-a33b-432c-9d7f-a17481e0d684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:19.816646 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c507e4e-a33b-432c-9d7f-a17481e0d684 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.816938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c507e4e-a33b-432c-9d7f-a17481e0d684 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.817205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c507e4e-a33b-432c-9d7f-a17481e0d684 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.817891 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c507e4e-a33b-432c-9d7f-a17481e0d684 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:19.819263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c507e4e-a33b-432c-9d7f-a17481e0d684 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:19.819980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c50442-6aba-454d-878f-3c8f2aee5306 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:19.830272 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:19.832656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-0c507e4e-a33b-432c-9d7f-a17481e0d684 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 15:35:19.832656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:19.832656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:19.832656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:19.832656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:19.832656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:19.832656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:19.832656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:19.832656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 15:35:19.832656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:19.832656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:19.832656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:19.832656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:19.832656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:19.832656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:19.832656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:19.832656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:19.833779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:19.833779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:19.833779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:19.833779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:19.833779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:19.833779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:19.833779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:19.833779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:19.833779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:19.833779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:19.833779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:19.833779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:19.833779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:19.833779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:19.833779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:19.833779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:19.835340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:19.835340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:19.835340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:19.835340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:19.835340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:19.835340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:19.835340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:19.835340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:19.835340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:19.835340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:19.835340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:19.835340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:19.835340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 15:35:19.835340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:19.838876 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 431/836] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:35:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:19.839897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c507e4e-a33b-432c-9d7f-a17481e0d684 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:19.864654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4f07a-c185-4c2b-8ad5-f6367af39e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:19.865549 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4f07a-c185-4c2b-8ad5-f6367af39e00 None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:19.865869 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4f07a-c185-4c2b-8ad5-f6367af39e00 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:19.866167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4f07a-c185-4c2b-8ad5-f6367af39e00 None admin] PATH_INFO: `/v3/projects/7be83fb6f20341139147373837320148/users/6655366e7eb84a26aa3f7fbe88af62b5/roles/805851682adf4debbe99b1681c9fcdfc` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:19.881997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4f07a-c185-4c2b-8ad5-f6367af39e00 None admin] RBAC: Authorizing `identity:create_grant(project_id=7be83fb6f20341139147373837320148, user_id=6655366e7eb84a26aa3f7fbe88af62b5, role_id=805851682adf4debbe99b1681c9fcdfc)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:19.883472 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4f07a-c185-4c2b-8ad5-f6367af39e00 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:19.897341 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 406/837] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:35:19 2022] PUT /identity/v3/projects/7be83fb6f20341139147373837320148/users/6655366e7eb84a26aa3f7fbe88af62b5/roles/805851682adf4debbe99b1681c9fcdfc => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:35:19.898466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4f07a-c185-4c2b-8ad5-f6367af39e00 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:19.971636 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c507e4e-a33b-432c-9d7f-a17481e0d684 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.015000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ea48d-fd8d-4e41-9af3-0020321ae21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.015808 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ea48d-fd8d-4e41-9af3-0020321ae21e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.016085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ea48d-fd8d-4e41-9af3-0020321ae21e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.016356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ea48d-fd8d-4e41-9af3-0020321ae21e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.029402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ea48d-fd8d-4e41-9af3-0020321ae21e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:20.037568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-144ea48d-fd8d-4e41-9af3-0020321ae21e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:35:20.037568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:20.037568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:20.037568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:20.037568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:20.037568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:20.037568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:20.037568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:20.037568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:20.037568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:20.037568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:20.037568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:20.037568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:20.037568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:20.037568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:20.037568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:20.038638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:20.038638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:20.038638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:20.038638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:20.038638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:20.038638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:20.038638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:20.038638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:20.038638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:20.038638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:20.038638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:20.038638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:20.038638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:20.038638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:20.038638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:20.038638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:20.040545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:20.040545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:20.040545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:20.040545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:20.040545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:20.040545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:20.040545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:20.040545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:20.040545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:20.040545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:20.040545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:35:20.040545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:20.045124 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 432/838] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:35:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:20.046144 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ea48d-fd8d-4e41-9af3-0020321ae21e None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.052855 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4f07a-c185-4c2b-8ad5-f6367af39e00 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.084289 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:20.120891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905e9aa-0c73-41e2-865b-c156a9057fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.121576 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905e9aa-0c73-41e2-865b-c156a9057fb9 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.122675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905e9aa-0c73-41e2-865b-c156a9057fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.122953 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905e9aa-0c73-41e2-865b-c156a9057fb9 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.123671 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905e9aa-0c73-41e2-865b-c156a9057fb9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:20.123974 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905e9aa-0c73-41e2-865b-c156a9057fb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.125376 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905e9aa-0c73-41e2-865b-c156a9057fb9 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.130766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 407/839] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:35:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:20.136743 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905e9aa-0c73-41e2-865b-c156a9057fb9 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.157591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:20.180881 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ea48d-fd8d-4e41-9af3-0020321ae21e None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.223029 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547eba66-4d92-4e46-a54a-4be71583c600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.223964 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547eba66-4d92-4e46-a54a-4be71583c600 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.224403 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547eba66-4d92-4e46-a54a-4be71583c600 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.224831 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547eba66-4d92-4e46-a54a-4be71583c600 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.225827 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547eba66-4d92-4e46-a54a-4be71583c600 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:20.226300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547eba66-4d92-4e46-a54a-4be71583c600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.228131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547eba66-4d92-4e46-a54a-4be71583c600 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.238699 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 433/840] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:35:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:20.245414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547eba66-4d92-4e46-a54a-4be71583c600 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.348968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:20.356519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547eba66-4d92-4e46-a54a-4be71583c600 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.378709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905e9aa-0c73-41e2-865b-c156a9057fb9 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.392571 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9fadd-3970-4a8d-8350-81cdadafd1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.393369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9fadd-3970-4a8d-8350-81cdadafd1ab None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.393665 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9fadd-3970-4a8d-8350-81cdadafd1ab None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.393975 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9fadd-3970-4a8d-8350-81cdadafd1ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.407027 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9fadd-3970-4a8d-8350-81cdadafd1ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.408399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9fadd-3970-4a8d-8350-81cdadafd1ab None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.420046 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4c46b-46c0-44c5-9e26-48e44590c5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.421047 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4c46b-46c0-44c5-9e26-48e44590c5ab None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.421502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4c46b-46c0-44c5-9e26-48e44590c5ab None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.421945 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4c46b-46c0-44c5-9e26-48e44590c5ab None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.422817 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4c46b-46c0-44c5-9e26-48e44590c5ab None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:20.423277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4c46b-46c0-44c5-9e26-48e44590c5ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.424912 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4c46b-46c0-44c5-9e26-48e44590c5ab None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.427441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-31c9fadd-3970-4a8d-8350-81cdadafd1ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:20.427441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:20.427441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:20.427441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:20.427441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:20.427441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:20.427441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:20.427441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:20.427441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:20.427441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:20.427441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:20.427441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:20.427441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:20.427441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:20.427441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:20.428634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:20.428634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:20.428634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:20.428634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:20.428634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:20.428634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:20.428634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:20.428634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:20.428634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:20.428634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:20.428634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:20.428634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:20.428634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:20.428634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:20.428634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:20.428634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:20.428634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:20.429739 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:20.429739 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:20.429739 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:20.429739 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:20.434365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 434/841] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:20.435372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9fadd-3970-4a8d-8350-81cdadafd1ab None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.441071 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 408/842] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:35:20 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:20.562158 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9fadd-3970-4a8d-8350-81cdadafd1ab None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.605288 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf694e11-4955-48fa-8ade-b80bc27a5e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.606066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf694e11-4955-48fa-8ade-b80bc27a5e8e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.606306 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf694e11-4955-48fa-8ade-b80bc27a5e8e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.606564 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf694e11-4955-48fa-8ade-b80bc27a5e8e None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.607308 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf694e11-4955-48fa-8ade-b80bc27a5e8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:20.607618 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf694e11-4955-48fa-8ade-b80bc27a5e8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.609005 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf694e11-4955-48fa-8ade-b80bc27a5e8e None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.623830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 435/843] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:20.631275 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4c46b-46c0-44c5-9e26-48e44590c5ab None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.730989 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:20.751405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4c46b-46c0-44c5-9e26-48e44590c5ab None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.787310 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7eb828-b716-458c-b110-01b3e4957b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.788116 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7eb828-b716-458c-b110-01b3e4957b9b None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.788400 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7eb828-b716-458c-b110-01b3e4957b9b None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.788675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7eb828-b716-458c-b110-01b3e4957b9b None admin] PATH_INFO: `/v3/users/swift` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.797414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7eb828-b716-458c-b110-01b3e4957b9b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.799023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7eb828-b716-458c-b110-01b3e4957b9b None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:20.814065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-3b7eb828-b716-458c-b110-01b3e4957b9b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:35:20.814065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:20.814065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:20.814065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:20.814065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:20.814065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:20.814065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:20.814065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:20.814065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:20.814065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:20.814065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:20.814065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:20.814065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:35:20.814065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:35:20.814065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:35:20.814065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:35:20.815032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:20.815032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:20.815032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:20.815032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:20.815032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:20.815032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:20.815032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:20.815032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:20.815032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:20.815032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:20.815032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:20.815032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:20.815032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:20.815032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:20.815032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:20.815032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:20.815032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:20.815032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:20.815967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:20.815967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:20.815967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:35:20.815967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:35:20.815967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:35:20.815967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:35:20.815967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:35:20.815967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:35:20.815967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 15:35:20.815967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:20.815967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 409/844] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:35:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:20.829176 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf694e11-4955-48fa-8ade-b80bc27a5e8e None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:20.838990 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:20.940243 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf694e11-4955-48fa-8ade-b80bc27a5e8e None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:20.977528 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd313e-6f06-4238-8eb8-d17cddab53be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:20.978629 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd313e-6f06-4238-8eb8-d17cddab53be None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:20.979179 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd313e-6f06-4238-8eb8-d17cddab53be None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:20.979657 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd313e-6f06-4238-8eb8-d17cddab53be None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:20.980700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd313e-6f06-4238-8eb8-d17cddab53be None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:20.981376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd313e-6f06-4238-8eb8-d17cddab53be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:20.983052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd313e-6f06-4238-8eb8-d17cddab53be None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:21.002696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 436/845] 10.222.0.20 () {56 vars in 1135 bytes} [Mon Jun 27 15:35:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:21.007812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57a7a7-150a-4276-9500-797b0a4361f0 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:21.008097 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57a7a7-150a-4276-9500-797b0a4361f0 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:21.008385 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57a7a7-150a-4276-9500-797b0a4361f0 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:21.009816 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd313e-6f06-4238-8eb8-d17cddab53be None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:21.010705 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 410/846] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:21.016273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15843fb5-bd57-4b17-9e4d-b9378b056069 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:21.016570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15843fb5-bd57-4b17-9e4d-b9378b056069 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:21.016836 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15843fb5-bd57-4b17-9e4d-b9378b056069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:21.035132 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-15843fb5-bd57-4b17-9e4d-b9378b056069 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:21.036459 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-15843fb5-bd57-4b17-9e4d-b9378b056069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:21.081084 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:21.121989 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 411/847] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:21.122970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd313e-6f06-4238-8eb8-d17cddab53be None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:21.136873 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15843fb5-bd57-4b17-9e4d-b9378b056069 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:21.161221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c085a0e-1fcd-4624-a0c8-decd7572f6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:21.162310 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c085a0e-1fcd-4624-a0c8-decd7572f6d5 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:21.162833 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c085a0e-1fcd-4624-a0c8-decd7572f6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:21.163318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c085a0e-1fcd-4624-a0c8-decd7572f6d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:21.222065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:21.238627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c085a0e-1fcd-4624-a0c8-decd7572f6d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:21.240408 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c085a0e-1fcd-4624-a0c8-decd7572f6d5 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:21.247830 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15843fb5-bd57-4b17-9e4d-b9378b056069 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:21.285007 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68e836-06d7-45a7-8813-f83d93513bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:21.285861 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68e836-06d7-45a7-8813-f83d93513bd6 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:21.286148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68e836-06d7-45a7-8813-f83d93513bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:21.286427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68e836-06d7-45a7-8813-f83d93513bd6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:21.287105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68e836-06d7-45a7-8813-f83d93513bd6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:21.288592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68e836-06d7-45a7-8813-f83d93513bd6 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:21.302261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-0f68e836-06d7-45a7-8813-f83d93513bd6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 15:35:21.302261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:21.302261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:21.302261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:21.302261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:21.302261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:21.302261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:21.302261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:21.302261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 15:35:21.302261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:21.302261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:21.302261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:21.302261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:21.302261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:21.302261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:21.302261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:21.303208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:21.303208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:21.303208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:21.303208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:21.303208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:21.303208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:21.303208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:21.303208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:21.303208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:21.303208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:21.303208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:21.303208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:21.303208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:21.303208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:21.303208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:21.303208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:21.304063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:21.304063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:21.304063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:21.304063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:21.304063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:21.304063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:21.304063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:21.304063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:21.304063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:21.304063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:21.304063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:21.304063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:21.304063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:21.304063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 15:35:21.304063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:21.308645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 412/848] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Jun 27 15:35:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:21.309834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68e836-06d7-45a7-8813-f83d93513bd6 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:21.313030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:21.362362 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 437/849] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:35:21 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 15:35:21.363453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c085a0e-1fcd-4624-a0c8-decd7572f6d5 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:21.423439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68e836-06d7-45a7-8813-f83d93513bd6 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:21.461737 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad9eb5-4084-45c3-b560-45f09102e373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:21.462699 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad9eb5-4084-45c3-b560-45f09102e373 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:21.463134 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad9eb5-4084-45c3-b560-45f09102e373 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:21.463580 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad9eb5-4084-45c3-b560-45f09102e373 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:21.464457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad9eb5-4084-45c3-b560-45f09102e373 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:21.465030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad9eb5-4084-45c3-b560-45f09102e373 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:21.466647 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad9eb5-4084-45c3-b560-45f09102e373 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:21.469025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:21.478145 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 413/850] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jun 27 15:35:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:21.479326 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad9eb5-4084-45c3-b560-45f09102e373 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:21.483046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c085a0e-1fcd-4624-a0c8-decd7572f6d5 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:21.523975 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b154b8-9195-4ce7-863e-8216eff1a923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:21.525936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b154b8-9195-4ce7-863e-8216eff1a923 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:21.526427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b154b8-9195-4ce7-863e-8216eff1a923 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:21.526904 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b154b8-9195-4ce7-863e-8216eff1a923 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:21.539530 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b154b8-9195-4ce7-863e-8216eff1a923 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:21.541549 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b154b8-9195-4ce7-863e-8216eff1a923 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:21.563838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-48b154b8-9195-4ce7-863e-8216eff1a923 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:21.563838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:21.563838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:21.563838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:21.563838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:21.563838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:21.563838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:21.563838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:21.563838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:21.563838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:21.563838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:21.563838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:21.563838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:21.563838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:21.563838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:21.564799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:21.564799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:21.564799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:21.564799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:21.564799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:21.564799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:21.564799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:21.564799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:21.564799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:21.564799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:21.564799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:21.564799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:21.564799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:21.564799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:21.564799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:21.564799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:21.564799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:21.566180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:21.566180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:21.566180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:21.566180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:21.568309 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 438/851] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:21.574850 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b154b8-9195-4ce7-863e-8216eff1a923 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:21.593134 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad9eb5-4084-45c3-b560-45f09102e373 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:21.597824 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:21.635969 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91686c95-1620-4d39-8573-9ee7010a1ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:21.636942 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91686c95-1620-4d39-8573-9ee7010a1ba3 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:21.637455 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91686c95-1620-4d39-8573-9ee7010a1ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:21.637969 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91686c95-1620-4d39-8573-9ee7010a1ba3 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:21.638947 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91686c95-1620-4d39-8573-9ee7010a1ba3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:21.639461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91686c95-1620-4d39-8573-9ee7010a1ba3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:21.641136 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91686c95-1620-4d39-8573-9ee7010a1ba3 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:21.653954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 414/852] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jun 27 15:35:21 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:21.712980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b154b8-9195-4ce7-863e-8216eff1a923 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:21.759319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007cd7b-a789-4054-aa9a-7096ec522485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:21.760036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007cd7b-a789-4054-aa9a-7096ec522485 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:21.760322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007cd7b-a789-4054-aa9a-7096ec522485 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:21.760604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007cd7b-a789-4054-aa9a-7096ec522485 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:21.761356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007cd7b-a789-4054-aa9a-7096ec522485 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:21.761689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007cd7b-a789-4054-aa9a-7096ec522485 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:21.763109 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007cd7b-a789-4054-aa9a-7096ec522485 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:21.776827 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 439/853] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:21.782528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91686c95-1620-4d39-8573-9ee7010a1ba3 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:21.864929 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:21.907799 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91686c95-1620-4d39-8573-9ee7010a1ba3 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:21.943723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99df47b-b55e-4046-8e0d-508e93465a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:21.944532 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99df47b-b55e-4046-8e0d-508e93465a12 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:21.944819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99df47b-b55e-4046-8e0d-508e93465a12 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:21.945101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99df47b-b55e-4046-8e0d-508e93465a12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:21.952477 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99df47b-b55e-4046-8e0d-508e93465a12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:21.954035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99df47b-b55e-4046-8e0d-508e93465a12 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:21.969534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-b99df47b-b55e-4046-8e0d-508e93465a12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:21.969534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:21.969534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:21.969534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:21.969534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:21.969534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:21.969534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:21.969534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:21.969534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:21.969534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:21.969534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:21.969534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:21.969534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:21.969534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:21.969534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:21.970544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:21.970544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:21.970544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:21.970544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:21.970544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:21.970544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:21.970544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:21.970544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:21.970544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:21.970544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:21.970544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:21.970544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:21.970544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:21.970544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:21.970544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:21.970544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:21.970544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:21.971430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:21.971430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:21.971430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:21.971430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:21.971430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:21.971430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:21.971430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:21.971430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:21.971430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 415/854] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:21.976857 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007cd7b-a789-4054-aa9a-7096ec522485 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:22.068512 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:22.102315 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007cd7b-a789-4054-aa9a-7096ec522485 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:22.140782 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2ab65-f014-4f04-8ba9-27c27cab9c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:22.141478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2ab65-f014-4f04-8ba9-27c27cab9c40 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:22.141801 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2ab65-f014-4f04-8ba9-27c27cab9c40 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:22.142084 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2ab65-f014-4f04-8ba9-27c27cab9c40 None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:22.142911 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2ab65-f014-4f04-8ba9-27c27cab9c40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:22.143226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2ab65-f014-4f04-8ba9-27c27cab9c40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:22.144731 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2ab65-f014-4f04-8ba9-27c27cab9c40 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:22.155697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c209c-7f37-44ee-b47e-2f2582b0762e None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:22.156047 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c209c-7f37-44ee-b47e-2f2582b0762e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:22.156242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c209c-7f37-44ee-b47e-2f2582b0762e None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:22.159136 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 440/856] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:22.159348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 416/856] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:22.165776 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c209c-7f37-44ee-b47e-2f2582b0762e None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:22.166239 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e2de9-6f8d-42b2-a4e5-012210148e86 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:22.166471 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e2de9-6f8d-42b2-a4e5-012210148e86 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:22.166745 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e2de9-6f8d-42b2-a4e5-012210148e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:22.184464 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-f33e2de9-6f8d-42b2-a4e5-012210148e86 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:22.185683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f33e2de9-6f8d-42b2-a4e5-012210148e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:22.269826 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 441/857] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:22.288528 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e2de9-6f8d-42b2-a4e5-012210148e86 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:22.307258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c209c-7f37-44ee-b47e-2f2582b0762e None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:22.341561 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:22.357729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46e5e4-8282-4bd4-a3e2-269ac01406da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:22.358688 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46e5e4-8282-4bd4-a3e2-269ac01406da None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:22.358980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46e5e4-8282-4bd4-a3e2-269ac01406da None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:22.359263 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46e5e4-8282-4bd4-a3e2-269ac01406da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:22.360796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46e5e4-8282-4bd4-a3e2-269ac01406da None admin] RBAC: Adding query filter params (role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.project.id=7be83fb6f20341139147373837320148, user.id=8e83ea69ded040448fe75caa221ad377) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:22.361120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46e5e4-8282-4bd4-a3e2-269ac01406da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:22.362883 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46e5e4-8282-4bd4-a3e2-269ac01406da None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:22.364514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:22.382310 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 417/858] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:35:22 2022] GET /identity/v3/role_assignments?user.id=8e83ea69ded040448fe75caa221ad377&scope.project.id=7be83fb6f20341139147373837320148&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:22.412661 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e2de9-6f8d-42b2-a4e5-012210148e86 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:22.453970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7f854-6925-4216-a7bc-30a6e4d5e43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:22.454806 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7f854-6925-4216-a7bc-30a6e4d5e43c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:22.455085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7f854-6925-4216-a7bc-30a6e4d5e43c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:22.455354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7f854-6925-4216-a7bc-30a6e4d5e43c None admin] PATH_INFO: `/v3/roles/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:22.468131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7f854-6925-4216-a7bc-30a6e4d5e43c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:22.476520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-edf7f854-6925-4216-a7bc-30a6e4d5e43c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:35:22.476520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:22.476520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:22.476520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:22.476520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:22.476520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:22.476520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:22.476520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:22.476520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:22.476520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:22.476520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:22.476520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:22.476520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:22.476520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:22.476520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:22.476520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:22.477626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:22.477626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:22.477626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:22.477626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:22.477626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:22.477626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:22.477626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:22.477626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:22.477626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:22.477626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:22.477626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:22.477626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:22.477626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:22.477626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:22.477626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:22.477626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:22.478557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:22.478557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:22.478557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:22.478557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:22.478557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:22.478557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:22.478557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:22.478557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:22.478557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:22.478557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:22.478557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:35:22.478557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:22.483334 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 442/859] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:35:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:22.484938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46e5e4-8282-4bd4-a3e2-269ac01406da None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:22.616889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46e5e4-8282-4bd4-a3e2-269ac01406da None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:22.654571 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112c4ed-3485-403e-825f-c30aa228944a None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:22.654887 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112c4ed-3485-403e-825f-c30aa228944a None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:22.655135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112c4ed-3485-403e-825f-c30aa228944a None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:22.656822 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 443/860] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:22.659832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f6698-0865-468e-ba66-cb536f490704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:22.660946 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f6698-0865-468e-ba66-cb536f490704 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:22.661445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f6698-0865-468e-ba66-cb536f490704 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:22.661964 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f6698-0865-468e-ba66-cb536f490704 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:22.662232 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc55fa-2020-4e27-bb7d-1d07c638abc3 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:22.662526 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc55fa-2020-4e27-bb7d-1d07c638abc3 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:22.662865 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc55fa-2020-4e27-bb7d-1d07c638abc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:22.663658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f6698-0865-468e-ba66-cb536f490704 None admin] RBAC: Adding query filter params (name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:22.664473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f6698-0865-468e-ba66-cb536f490704 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:22.666579 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f6698-0865-468e-ba66-cb536f490704 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:22.682252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 418/861] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:35:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:22.687685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-e5cc55fa-2020-4e27-bb7d-1d07c638abc3 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:22.688907 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e5cc55fa-2020-4e27-bb7d-1d07c638abc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:22.689196 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f6698-0865-468e-ba66-cb536f490704 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:22.748669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:22.773933 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 444/862] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:35:22.802573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc55fa-2020-4e27-bb7d-1d07c638abc3 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:22.827997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f6698-0865-468e-ba66-cb536f490704 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:22.868114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0c2f2-302c-4ea9-ad72-998a858fdcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:22.868637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:22.869605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0c2f2-302c-4ea9-ad72-998a858fdcf4 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:22.870033 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0c2f2-302c-4ea9-ad72-998a858fdcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:22.870370 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0c2f2-302c-4ea9-ad72-998a858fdcf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:22.885891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0c2f2-302c-4ea9-ad72-998a858fdcf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:22.887702 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0c2f2-302c-4ea9-ad72-998a858fdcf4 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:22.908181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-40e0c2f2-302c-4ea9-ad72-998a858fdcf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:22.908181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:22.908181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:22.908181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:22.908181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:22.908181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:22.908181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:22.908181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:22.908181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:22.908181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:22.908181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:22.908181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:22.908181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:22.908181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:22.908181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:22.909178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:22.909178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:22.909178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:22.909178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:22.909178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:22.909178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:22.909178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:22.909178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:22.909178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:22.909178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:22.909178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:22.909178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:22.909178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:22.909178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:22.909178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:22.909178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:22.909178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:22.910110 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:22.910110 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:22.910110 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:22.910110 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:22.913791 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 419/863] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:22.916351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0c2f2-302c-4ea9-ad72-998a858fdcf4 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:22.936026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc55fa-2020-4e27-bb7d-1d07c638abc3 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:22.986835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc51d9f6-b50d-40df-a421-88bcbb752eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:22.987962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51d9f6-b50d-40df-a421-88bcbb752eb9 None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:22.988468 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51d9f6-b50d-40df-a421-88bcbb752eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:22.988954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51d9f6-b50d-40df-a421-88bcbb752eb9 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:22.990085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc51d9f6-b50d-40df-a421-88bcbb752eb9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:22.991690 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc51d9f6-b50d-40df-a421-88bcbb752eb9 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:23.013992 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 445/864] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jun 27 15:35:22 2022] POST /identity/v3/services => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:35:23.043470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0c2f2-302c-4ea9-ad72-998a858fdcf4 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:23.045245 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:23.090966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ded32-8948-45e0-864a-869a6d3b499f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:23.091650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ded32-8948-45e0-864a-869a6d3b499f None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:23.091932 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ded32-8948-45e0-864a-869a6d3b499f None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:23.092205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ded32-8948-45e0-864a-869a6d3b499f None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:23.092941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ded32-8948-45e0-864a-869a6d3b499f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:23.093245 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ded32-8948-45e0-864a-869a6d3b499f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:23.094685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ded32-8948-45e0-864a-869a6d3b499f None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:23.113153 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 420/865] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:23.118826 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc51d9f6-b50d-40df-a421-88bcbb752eb9 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:23.235935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc51d9f6-b50d-40df-a421-88bcbb752eb9 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:23.256526 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:23.285184 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d23177-d8ad-499a-8679-d95cd8aa4b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:23.286438 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d23177-d8ad-499a-8679-d95cd8aa4b90 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:23.286951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d23177-d8ad-499a-8679-d95cd8aa4b90 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:23.287446 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d23177-d8ad-499a-8679-d95cd8aa4b90 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:23.296781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d23177-d8ad-499a-8679-d95cd8aa4b90 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:23.298580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d23177-d8ad-499a-8679-d95cd8aa4b90 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:23.314939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-50d23177-d8ad-499a-8679-d95cd8aa4b90 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:35:23.314939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:23.314939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:23.314939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:23.314939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:23.314939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:23.314939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:23.314939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:23.314939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:23.314939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:23.314939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:23.314939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:23.314939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:35:23.314939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:35:23.314939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:35:23.314939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:35:23.315926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:23.315926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:23.315926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:23.315926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:23.315926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:23.315926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:23.315926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:23.315926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:23.315926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:23.315926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:23.315926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:23.315926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:23.315926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:23.315926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:23.315926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:23.315926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:23.315926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:23.315926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:23.316849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:23.316849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:23.316849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:35:23.316849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:35:23.316849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:35:23.316849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:35:23.316849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:35:23.316849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:35:23.316849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:35:23.316849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:23.319542 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 446/866] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jun 27 15:35:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:23.326242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ded32-8948-45e0-864a-869a6d3b499f None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:23.434810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ded32-8948-45e0-864a-869a6d3b499f None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:23.473122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033eeb1-4965-4824-811e-de88e969d712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:23.473839 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033eeb1-4965-4824-811e-de88e969d712 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:23.474121 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033eeb1-4965-4824-811e-de88e969d712 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:23.474394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033eeb1-4965-4824-811e-de88e969d712 None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:23.475236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033eeb1-4965-4824-811e-de88e969d712 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:23.475541 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033eeb1-4965-4824-811e-de88e969d712 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:23.476968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033eeb1-4965-4824-811e-de88e969d712 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:23.501167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 421/867] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Jun 27 15:35:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:23.502672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d23177-d8ad-499a-8679-d95cd8aa4b90 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:23.615770 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d23177-d8ad-499a-8679-d95cd8aa4b90 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:23.655421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceec5da-f478-4d6e-bc28-5aa1eba71a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:23.656097 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceec5da-f478-4d6e-bc28-5aa1eba71a52 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:23.656388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceec5da-f478-4d6e-bc28-5aa1eba71a52 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:23.656670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceec5da-f478-4d6e-bc28-5aa1eba71a52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:23.705801 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:23.731689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceec5da-f478-4d6e-bc28-5aa1eba71a52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:23.733139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceec5da-f478-4d6e-bc28-5aa1eba71a52 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:23.849423 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 447/868] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:35:23 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:35:23.850836 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033eeb1-4965-4824-811e-de88e969d712 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:23.870300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:23.884418 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6627f0-1c80-4531-8afb-51eec383a36e None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:23.884696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6627f0-1c80-4531-8afb-51eec383a36e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:23.884953 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6627f0-1c80-4531-8afb-51eec383a36e None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:23.886730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 448/869] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:23.891953 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa0c6d-b16e-4167-9296-cbc5de1ee92b None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:23.892240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa0c6d-b16e-4167-9296-cbc5de1ee92b None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:23.892593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa0c6d-b16e-4167-9296-cbc5de1ee92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:23.911724 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-66fa0c6d-b16e-4167-9296-cbc5de1ee92b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:23.912932 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-66fa0c6d-b16e-4167-9296-cbc5de1ee92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:24.000110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 449/870] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:23 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:24.000310 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033eeb1-4965-4824-811e-de88e969d712 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:24.022877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa0c6d-b16e-4167-9296-cbc5de1ee92b None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:24.032639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:24.048568 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a340597-14c4-457d-903d-5613d0eb101b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:24.050255 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a340597-14c4-457d-903d-5613d0eb101b None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:24.050978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a340597-14c4-457d-903d-5613d0eb101b None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:24.051463 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a340597-14c4-457d-903d-5613d0eb101b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:24.067843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a340597-14c4-457d-903d-5613d0eb101b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:24.070590 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a340597-14c4-457d-903d-5613d0eb101b None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:24.091654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-6a340597-14c4-457d-903d-5613d0eb101b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:24.091654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:24.091654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:24.091654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:24.091654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:24.091654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:24.091654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:24.091654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:24.091654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:24.091654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:24.091654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:24.091654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:24.091654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:24.091654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:24.091654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:24.092619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:24.092619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:24.092619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:24.092619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:24.092619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:24.092619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:24.092619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:24.092619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:24.092619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:24.092619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:24.092619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:24.092619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:24.092619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:24.092619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:24.092619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:24.092619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:24.092619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:24.093502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:24.093502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:24.093502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:24.093502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:24.093502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 422/871] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:24.101689 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a340597-14c4-457d-903d-5613d0eb101b None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:24.151442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa0c6d-b16e-4167-9296-cbc5de1ee92b None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:24.192380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:24.193540 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086ecfc-0fcc-4ab2-9c76-48b3d9656b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:24.194899 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086ecfc-0fcc-4ab2-9c76-48b3d9656b21 None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:24.195191 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086ecfc-0fcc-4ab2-9c76-48b3d9656b21 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:24.195469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086ecfc-0fcc-4ab2-9c76-48b3d9656b21 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:24.197509 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086ecfc-0fcc-4ab2-9c76-48b3d9656b21 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:24.199119 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086ecfc-0fcc-4ab2-9c76-48b3d9656b21 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:24.217276 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 450/872] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jun 27 15:35:24 2022] POST /identity/v3/services => generated 269 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:35:24.238826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a340597-14c4-457d-903d-5613d0eb101b None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:24.290852 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e45aa-f307-491f-925b-b454f31b2a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:24.291825 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e45aa-f307-491f-925b-b454f31b2a81 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:24.292257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e45aa-f307-491f-925b-b454f31b2a81 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:24.292722 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e45aa-f307-491f-925b-b454f31b2a81 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:24.293777 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e45aa-f307-491f-925b-b454f31b2a81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:24.294251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e45aa-f307-491f-925b-b454f31b2a81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:24.295948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e45aa-f307-491f-925b-b454f31b2a81 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:24.315890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 423/873] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:24.318181 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086ecfc-0fcc-4ab2-9c76-48b3d9656b21 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:24.405466 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:24.449679 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086ecfc-0fcc-4ab2-9c76-48b3d9656b21 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:24.490198 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d87bf-d580-4522-962e-355d12732df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:24.490947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d87bf-d580-4522-962e-355d12732df3 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:24.491233 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d87bf-d580-4522-962e-355d12732df3 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:24.491509 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d87bf-d580-4522-962e-355d12732df3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:24.501672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d87bf-d580-4522-962e-355d12732df3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:24.503731 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d87bf-d580-4522-962e-355d12732df3 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:24.520620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-803d87bf-d580-4522-962e-355d12732df3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:24.520620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:24.520620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:24.520620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:24.520620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:24.520620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:24.520620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:24.520620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:24.520620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:24.520620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:24.520620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:24.520620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:24.520620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:24.520620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:24.520620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:24.521570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:24.521570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:24.521570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:24.521570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:24.521570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:24.521570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:24.521570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:24.521570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:24.521570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:24.521570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:24.521570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:24.521570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:24.521570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:24.521570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:24.521570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:24.521570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:24.521570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:24.522467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:24.522467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:24.522467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:24.522467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:24.522467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:24.522467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:24.522467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:24.522467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:24.529794 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 451/874] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:24.531442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e45aa-f307-491f-925b-b454f31b2a81 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:24.656845 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e45aa-f307-491f-925b-b454f31b2a81 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:24.698923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:24.728821 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f4234-9db2-4849-8ed7-c895bd13e6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:24.729514 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f4234-9db2-4849-8ed7-c895bd13e6ff None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:24.729829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f4234-9db2-4849-8ed7-c895bd13e6ff None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:24.730107 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f4234-9db2-4849-8ed7-c895bd13e6ff None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:24.730918 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f4234-9db2-4849-8ed7-c895bd13e6ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:24.731227 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f4234-9db2-4849-8ed7-c895bd13e6ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:24.732686 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f4234-9db2-4849-8ed7-c895bd13e6ff None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:24.735656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b380c4f-7edb-4b92-9843-409973c1dc7a None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:24.735948 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b380c4f-7edb-4b92-9843-409973c1dc7a None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:24.736227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b380c4f-7edb-4b92-9843-409973c1dc7a None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:24.746380 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 452/875] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:24.748496 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3efd4-9ac1-478a-a89c-6c41d44b70c6 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:24.748867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3efd4-9ac1-478a-a89c-6c41d44b70c6 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:24.749424 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3efd4-9ac1-478a-a89c-6c41d44b70c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:24.767468 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 424/876] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:24.773754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f4234-9db2-4849-8ed7-c895bd13e6ff None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:24.777621 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-e1e3efd4-9ac1-478a-a89c-6c41d44b70c6 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:24.780074 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e1e3efd4-9ac1-478a-a89c-6c41d44b70c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:24.906194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 453/877] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:24.926083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3efd4-9ac1-478a-a89c-6c41d44b70c6 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:24.996854 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f4234-9db2-4849-8ed7-c895bd13e6ff None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:25.008297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:25.052017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54a70f-1497-412b-82e8-8a484127e957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:25.053117 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54a70f-1497-412b-82e8-8a484127e957 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:25.053630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54a70f-1497-412b-82e8-8a484127e957 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:25.054143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54a70f-1497-412b-82e8-8a484127e957 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:25.055208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54a70f-1497-412b-82e8-8a484127e957 None admin] RBAC: Adding query filter params (role.id=805851682adf4debbe99b1681c9fcdfc, scope.project.id=7be83fb6f20341139147373837320148, user.id=6655366e7eb84a26aa3f7fbe88af62b5) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:25.056269 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54a70f-1497-412b-82e8-8a484127e957 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:25.057975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54a70f-1497-412b-82e8-8a484127e957 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:25.064980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 425/878] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:35:24 2022] GET /identity/v3/role_assignments?user.id=6655366e7eb84a26aa3f7fbe88af62b5&scope.project.id=7be83fb6f20341139147373837320148&role.id=805851682adf4debbe99b1681c9fcdfc => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:25.066500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3efd4-9ac1-478a-a89c-6c41d44b70c6 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:25.113021 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4422c28-9773-45dd-a896-bd5a2ff6fe22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:25.114297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4422c28-9773-45dd-a896-bd5a2ff6fe22 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:25.114811 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4422c28-9773-45dd-a896-bd5a2ff6fe22 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:25.115810 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4422c28-9773-45dd-a896-bd5a2ff6fe22 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:25.116730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4422c28-9773-45dd-a896-bd5a2ff6fe22 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:25.118951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4422c28-9773-45dd-a896-bd5a2ff6fe22 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:25.137360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-b4422c28-9773-45dd-a896-bd5a2ff6fe22 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:35:25.137360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:25.137360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:25.137360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:25.137360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:25.137360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:25.137360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:25.137360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:25.137360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:35:25.137360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:25.137360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:25.137360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:25.137360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:25.137360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:25.137360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:25.137360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:25.137360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:25.138536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:25.138536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:25.138536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:25.138536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:25.138536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:25.138536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:25.138536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:25.138536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:25.138536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:25.138536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:25.138536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:25.138536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:25.138536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:25.138536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:25.138536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:25.138536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:25.139625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:25.139625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:25.139625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:25.139625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:25.139625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:25.139625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:25.139625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:25.139625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:25.139625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:25.139625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:25.139625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:25.139625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:25.139625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:35:25.139625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:25.144455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 454/879] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jun 27 15:35:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:25.151090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54a70f-1497-412b-82e8-8a484127e957 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:25.211215 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:25.276137 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54a70f-1497-412b-82e8-8a484127e957 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:25.314089 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73dba7-cced-451c-96cc-e00ac2f822de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:25.314756 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73dba7-cced-451c-96cc-e00ac2f822de None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:25.315035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73dba7-cced-451c-96cc-e00ac2f822de None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:25.315310 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73dba7-cced-451c-96cc-e00ac2f822de None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:25.316082 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73dba7-cced-451c-96cc-e00ac2f822de None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:25.316388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73dba7-cced-451c-96cc-e00ac2f822de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:25.317852 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73dba7-cced-451c-96cc-e00ac2f822de None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:25.324001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 426/880] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jun 27 15:35:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:25.329984 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4422c28-9773-45dd-a896-bd5a2ff6fe22 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:25.359563 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b538a-918c-4665-9289-3ff845ca8243 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:25.360119 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b538a-918c-4665-9289-3ff845ca8243 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:25.360410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b538a-918c-4665-9289-3ff845ca8243 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:25.363266 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 427/881] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:25.368574 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019af57-ab0b-4950-aeb4-96410cf38422 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:25.369033 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019af57-ab0b-4950-aeb4-96410cf38422 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:25.369409 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019af57-ab0b-4950-aeb4-96410cf38422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:25.390778 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-d019af57-ab0b-4950-aeb4-96410cf38422 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:25.392113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d019af57-ab0b-4950-aeb4-96410cf38422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:25.465573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4422c28-9773-45dd-a896-bd5a2ff6fe22 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:25.484311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 428/882] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:25.489948 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:25.507737 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019af57-ab0b-4950-aeb4-96410cf38422 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:25.509605 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab5a77-3d2f-46c0-ada1-319fdca63f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:25.510478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab5a77-3d2f-46c0-ada1-319fdca63f36 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:25.510957 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab5a77-3d2f-46c0-ada1-319fdca63f36 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:25.511426 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab5a77-3d2f-46c0-ada1-319fdca63f36 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:25.512188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab5a77-3d2f-46c0-ada1-319fdca63f36 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:25.512507 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab5a77-3d2f-46c0-ada1-319fdca63f36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:25.513923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab5a77-3d2f-46c0-ada1-319fdca63f36 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:25.519991 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 455/883] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jun 27 15:35:25 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:25.562668 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:25.632744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019af57-ab0b-4950-aeb4-96410cf38422 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:25.673657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81a944-47cb-427f-9cbd-4f380fa10659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:25.674613 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81a944-47cb-427f-9cbd-4f380fa10659 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:25.675219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81a944-47cb-427f-9cbd-4f380fa10659 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:25.675494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81a944-47cb-427f-9cbd-4f380fa10659 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:25.676233 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81a944-47cb-427f-9cbd-4f380fa10659 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:25.677654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81a944-47cb-427f-9cbd-4f380fa10659 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:25.691767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-2e81a944-47cb-427f-9cbd-4f380fa10659 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 15:35:25.691767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:25.691767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:25.691767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:25.691767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:25.691767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:25.691767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:25.691767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:25.691767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 15:35:25.691767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:25.691767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:25.691767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:25.691767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:25.691767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:25.691767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:25.691767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:25.691767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:25.693113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:25.693113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:25.693113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:25.693113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:25.693113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:25.693113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:25.693113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:25.693113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:25.693113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:25.693113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:25.693113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:25.693113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:25.693113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:25.693113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:25.693113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:25.693113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:25.694375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:25.694375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:25.694375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:25.694375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:25.694375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:25.694375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:25.694375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:25.694375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:25.694375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:25.694375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:25.694375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:25.694375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:25.694375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 15:35:25.694375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:25.694375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 429/884] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:35:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:25.699864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab5a77-3d2f-46c0-ada1-319fdca63f36 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:25.814497 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab5a77-3d2f-46c0-ada1-319fdca63f36 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:25.853285 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b211f37-361a-4145-aab7-36d23af179dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:25.854318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b211f37-361a-4145-aab7-36d23af179dc None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:25.854800 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b211f37-361a-4145-aab7-36d23af179dc None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:25.855331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b211f37-361a-4145-aab7-36d23af179dc None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:25.856233 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b211f37-361a-4145-aab7-36d23af179dc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:25.856751 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b211f37-361a-4145-aab7-36d23af179dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:25.858340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b211f37-361a-4145-aab7-36d23af179dc None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:25.864070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 456/885] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:35:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:25.869686 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81a944-47cb-427f-9cbd-4f380fa10659 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:25.980541 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81a944-47cb-427f-9cbd-4f380fa10659 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:25.994788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:26.017699 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c35a8-47bd-42bf-8031-985ed8822030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:26.018471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c35a8-47bd-42bf-8031-985ed8822030 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:26.018671 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c35a8-47bd-42bf-8031-985ed8822030 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:26.018954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c35a8-47bd-42bf-8031-985ed8822030 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:26.019680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c35a8-47bd-42bf-8031-985ed8822030 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:26.019992 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c35a8-47bd-42bf-8031-985ed8822030 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:26.021378 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c35a8-47bd-42bf-8031-985ed8822030 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:26.027471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 430/886] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:35:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:26.036656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b211f37-361a-4145-aab7-36d23af179dc None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:26.100665 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:26.157183 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b211f37-361a-4145-aab7-36d23af179dc None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:26.193263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb812fa4-596b-46c5-b840-4fbf1c57752b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:26.193966 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb812fa4-596b-46c5-b840-4fbf1c57752b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:26.194252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb812fa4-596b-46c5-b840-4fbf1c57752b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:26.194531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb812fa4-596b-46c5-b840-4fbf1c57752b None admin] PATH_INFO: `/v3/endpoints` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:26.195327 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb812fa4-596b-46c5-b840-4fbf1c57752b None admin] RBAC: Adding query filter params (interface=public, service_id=004caa593f4f428b91c986dba8077c3b, region_id=RegionOne) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:26.195645 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb812fa4-596b-46c5-b840-4fbf1c57752b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:26.197700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb812fa4-596b-46c5-b840-4fbf1c57752b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:26.203574 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 457/887] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Jun 27 15:35:26 2022] GET /identity/v3/endpoints?service_id=004caa593f4f428b91c986dba8077c3b&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:26.210758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c35a8-47bd-42bf-8031-985ed8822030 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:26.325679 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c35a8-47bd-42bf-8031-985ed8822030 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:26.362452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b36d9-0cfc-4f4c-8bbe-142a01396979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:26.363049 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b36d9-0cfc-4f4c-8bbe-142a01396979 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:26.363396 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b36d9-0cfc-4f4c-8bbe-142a01396979 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:26.363674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b36d9-0cfc-4f4c-8bbe-142a01396979 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:26.364415 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b36d9-0cfc-4f4c-8bbe-142a01396979 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:26.365887 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b36d9-0cfc-4f4c-8bbe-142a01396979 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:26.373471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 431/888] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Jun 27 15:35:26 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:35:26.626751 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b16311-3883-4236-992a-54dc5c41d922 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:26.627096 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b16311-3883-4236-992a-54dc5c41d922 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:26.627281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b16311-3883-4236-992a-54dc5c41d922 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:26.628848 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 458/889] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:26.634731 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc80be7-2e6b-4dc0-ac42-3c514e32716e None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:26.635026 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc80be7-2e6b-4dc0-ac42-3c514e32716e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:26.635310 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc80be7-2e6b-4dc0-ac42-3c514e32716e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:26.654050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-0bc80be7-2e6b-4dc0-ac42-3c514e32716e None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:26.655980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-0bc80be7-2e6b-4dc0-ac42-3c514e32716e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:26.691691 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:26.737113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 432/890] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:26.760197 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b16311-3883-4236-992a-54dc5c41d922 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:26.884327 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b16311-3883-4236-992a-54dc5c41d922 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:26.898844 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:26.926438 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a88caf-26b0-418d-aaa8-59626bc0518a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:26.927261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a88caf-26b0-418d-aaa8-59626bc0518a None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:26.927547 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a88caf-26b0-418d-aaa8-59626bc0518a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:26.927824 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a88caf-26b0-418d-aaa8-59626bc0518a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:26.928483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a88caf-26b0-418d-aaa8-59626bc0518a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:26.929954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a88caf-26b0-418d-aaa8-59626bc0518a None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:26.942936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-c1a88caf-26b0-418d-aaa8-59626bc0518a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 15:35:26.942936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:26.942936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:26.942936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:26.942936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:26.942936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:26.942936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:26.942936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:26.942936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 15:35:26.942936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:26.942936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:26.942936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:26.942936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:26.942936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:26.942936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:26.942936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:26.944148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:26.944148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:26.944148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:26.944148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:26.944148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:26.944148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:26.944148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:26.944148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:26.944148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:26.944148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:26.944148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:26.944148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:26.944148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:26.944148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:26.944148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:26.944148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:26.944956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:26.944956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:26.944956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:26.944956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:26.944956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:26.944956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:26.944956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:26.944956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:26.944956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:26.944956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:26.944956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:26.944956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:26.944956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:26.944956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 15:35:26.944956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:26.944956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 459/891] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Jun 27 15:35:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:26.952739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc80be7-2e6b-4dc0-ac42-3c514e32716e None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:27.057599 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc80be7-2e6b-4dc0-ac42-3c514e32716e None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:27.094400 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788b2f9-f5ee-4ac6-a2cc-a9a3d93db48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:27.095076 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788b2f9-f5ee-4ac6-a2cc-a9a3d93db48d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:27.095366 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788b2f9-f5ee-4ac6-a2cc-a9a3d93db48d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:27.095648 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788b2f9-f5ee-4ac6-a2cc-a9a3d93db48d None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:27.096365 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788b2f9-f5ee-4ac6-a2cc-a9a3d93db48d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:27.096678 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788b2f9-f5ee-4ac6-a2cc-a9a3d93db48d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:27.098105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788b2f9-f5ee-4ac6-a2cc-a9a3d93db48d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:27.104099 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 433/892] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jun 27 15:35:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:27.110733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a88caf-26b0-418d-aaa8-59626bc0518a None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:27.217301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a88caf-26b0-418d-aaa8-59626bc0518a None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:27.253140 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed8826-dff5-41bc-949c-9485a7997884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:27.257773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed8826-dff5-41bc-949c-9485a7997884 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:27.257773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed8826-dff5-41bc-949c-9485a7997884 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:27.257773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed8826-dff5-41bc-949c-9485a7997884 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:27.257773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed8826-dff5-41bc-949c-9485a7997884 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:27.257773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed8826-dff5-41bc-949c-9485a7997884 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:27.257773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed8826-dff5-41bc-949c-9485a7997884 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:27.264689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 460/893] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jun 27 15:35:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:27.270806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788b2f9-f5ee-4ac6-a2cc-a9a3d93db48d None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:27.308052 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:27.379416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788b2f9-f5ee-4ac6-a2cc-a9a3d93db48d None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:27.388948 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6728023-ee0d-4ae5-a4d1-6dc6a42af050 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:27.389235 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6728023-ee0d-4ae5-a4d1-6dc6a42af050 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:27.389514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6728023-ee0d-4ae5-a4d1-6dc6a42af050 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:27.391173 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 461/894] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:27.396258 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5412f-fb3b-4f35-8719-554e09f81b1b None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:27.396544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5412f-fb3b-4f35-8719-554e09f81b1b None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:27.396813 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5412f-fb3b-4f35-8719-554e09f81b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:27.421516 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-0ff5412f-fb3b-4f35-8719-554e09f81b1b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:27.422872 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0ff5412f-fb3b-4f35-8719-554e09f81b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:27.423693 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62b5d8-75a9-45f9-b156-b3aadc89dae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:27.424364 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62b5d8-75a9-45f9-b156-b3aadc89dae2 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:27.424647 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62b5d8-75a9-45f9-b156-b3aadc89dae2 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:27.424922 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62b5d8-75a9-45f9-b156-b3aadc89dae2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:27.425769 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62b5d8-75a9-45f9-b156-b3aadc89dae2 None admin] RBAC: Adding query filter params (interface=public, service_id=2c3621fca46744c9b4fc88cbb51aaa2e, region_id=RegionOne) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:27.426086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62b5d8-75a9-45f9-b156-b3aadc89dae2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:27.427466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62b5d8-75a9-45f9-b156-b3aadc89dae2 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:27.434305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 434/895] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Jun 27 15:35:27 2022] GET /identity/v3/endpoints?service_id=2c3621fca46744c9b4fc88cbb51aaa2e&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:27.439579 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62b5d8-75a9-45f9-b156-b3aadc89dae2 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:27.470242 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:27.518532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 462/896] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:35:27.546971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5412f-fb3b-4f35-8719-554e09f81b1b None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:27.568874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:27.580690 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62b5d8-75a9-45f9-b156-b3aadc89dae2 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:27.619878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b090a-5039-484f-ad91-342118ec20c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:27.621013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b090a-5039-484f-ad91-342118ec20c6 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:27.621576 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b090a-5039-484f-ad91-342118ec20c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:27.622099 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b090a-5039-484f-ad91-342118ec20c6 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:27.622998 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b090a-5039-484f-ad91-342118ec20c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:27.624588 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b090a-5039-484f-ad91-342118ec20c6 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:27.631767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 435/897] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Jun 27 15:35:27 2022] GET /identity/v3/services => generated 1268 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:35:27.672455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5412f-fb3b-4f35-8719-554e09f81b1b None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:27.714439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360a765-89a3-4966-b9ee-3bf1544f0296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:27.715653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360a765-89a3-4966-b9ee-3bf1544f0296 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:27.716130 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360a765-89a3-4966-b9ee-3bf1544f0296 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:27.716630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360a765-89a3-4966-b9ee-3bf1544f0296 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:27.729500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e360a765-89a3-4966-b9ee-3bf1544f0296 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:27.738514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-e360a765-89a3-4966-b9ee-3bf1544f0296 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:35:27.738514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:27.738514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:27.738514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:27.738514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:27.738514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:27.738514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:27.738514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:27.738514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:27.738514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:27.738514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:27.738514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:27.738514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:27.738514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:27.738514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:27.738514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:27.739481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:27.739481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:27.739481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:27.739481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:27.739481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:27.739481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:27.739481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:27.739481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:27.739481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:27.739481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:27.739481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:27.739481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:27.739481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:27.739481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:27.739481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:27.739481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:27.740303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:27.740303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:27.740303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:27.740303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:27.740303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:27.740303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:27.740303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:27.740303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:27.740303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:27.740303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:27.740303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:35:27.740303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:27.743690 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 463/898] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Jun 27 15:35:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:27.750007 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b090a-5039-484f-ad91-342118ec20c6 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:27.865367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b090a-5039-484f-ad91-342118ec20c6 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:27.865590 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd65420-500b-421c-9194-8fec27a558e0 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:27.865590 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd65420-500b-421c-9194-8fec27a558e0 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:27.865590 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd65420-500b-421c-9194-8fec27a558e0 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:27.869926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 464/899] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:27.871515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d830ba-106e-456e-9aba-598d73b32466 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:27.871950 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d830ba-106e-456e-9aba-598d73b32466 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:27.872371 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d830ba-106e-456e-9aba-598d73b32466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:27.890767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-80d830ba-106e-456e-9aba-598d73b32466 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:27.892413 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-80d830ba-106e-456e-9aba-598d73b32466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:27.905715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14c3af-6fc5-4bba-b514-71e5d450663a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:27.906422 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14c3af-6fc5-4bba-b514-71e5d450663a None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:27.906987 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14c3af-6fc5-4bba-b514-71e5d450663a None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:27.907437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf14c3af-6fc5-4bba-b514-71e5d450663a None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:27.908377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf14c3af-6fc5-4bba-b514-71e5d450663a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:27.909184 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf14c3af-6fc5-4bba-b514-71e5d450663a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:27.911554 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf14c3af-6fc5-4bba-b514-71e5d450663a None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:27.920988 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:27.923448 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 436/900] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Jun 27 15:35:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:27.928998 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14c3af-6fc5-4bba-b514-71e5d450663a None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:27.949991 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:27.989670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 465/901] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:35:28.011335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d830ba-106e-456e-9aba-598d73b32466 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:28.061854 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf14c3af-6fc5-4bba-b514-71e5d450663a None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:28.109800 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbae166-81fd-46f3-a065-23e3e742d5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:28.111393 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbae166-81fd-46f3-a065-23e3e742d5cb None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:28.112075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbae166-81fd-46f3-a065-23e3e742d5cb None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:28.112606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbae166-81fd-46f3-a065-23e3e742d5cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:28.126204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbae166-81fd-46f3-a065-23e3e742d5cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:28.127829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbae166-81fd-46f3-a065-23e3e742d5cb None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:28.130575 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d830ba-106e-456e-9aba-598d73b32466 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:28.151723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-4cbae166-81fd-46f3-a065-23e3e742d5cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:28.151723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:28.151723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:28.151723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:28.151723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:28.151723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:28.151723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:28.151723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:28.151723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:28.151723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:28.151723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:28.151723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:28.151723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:28.151723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:28.151723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:28.152773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:28.152773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:28.152773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:28.152773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:28.152773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:28.152773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:28.152773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:28.152773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:28.152773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:28.152773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:28.152773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:28.152773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:28.152773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:28.152773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:28.152773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:28.152773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:28.152773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:28.153710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:28.153710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:28.153710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:28.153710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:28.157719 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 437/902] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:28.159235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbae166-81fd-46f3-a065-23e3e742d5cb None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:28.189277 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a260b-e415-4346-b1fe-dd94f44b818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:28.190931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a260b-e415-4346-b1fe-dd94f44b818d None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:28.191395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a260b-e415-4346-b1fe-dd94f44b818d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:28.192102 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a260b-e415-4346-b1fe-dd94f44b818d None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:28.193438 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a260b-e415-4346-b1fe-dd94f44b818d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:28.195119 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a260b-e415-4346-b1fe-dd94f44b818d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:28.217676 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 466/903] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jun 27 15:35:28 2022] POST /identity/v3/services => generated 241 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:35:28.256850 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:28.296718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbae166-81fd-46f3-a065-23e3e742d5cb None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:28.344143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4faf5-0429-4682-9474-0e7d268f3c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:28.345147 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4faf5-0429-4682-9474-0e7d268f3c66 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:28.345589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4faf5-0429-4682-9474-0e7d268f3c66 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:28.346122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4faf5-0429-4682-9474-0e7d268f3c66 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:28.347081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4faf5-0429-4682-9474-0e7d268f3c66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:28.347553 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4faf5-0429-4682-9474-0e7d268f3c66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:28.349459 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4faf5-0429-4682-9474-0e7d268f3c66 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:28.365235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 438/904] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:28.370750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a260b-e415-4346-b1fe-dd94f44b818d None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:28.397212 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:28.492300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a260b-e415-4346-b1fe-dd94f44b818d None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:28.512785 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed276d-801c-477a-94da-9b6023f9b84e None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:28.512973 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed276d-801c-477a-94da-9b6023f9b84e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:28.513255 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed276d-801c-477a-94da-9b6023f9b84e None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:28.514801 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 439/905] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:28.520197 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7733c5b-4cd2-4055-970f-46bcf37dfb6f None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:28.520754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7733c5b-4cd2-4055-970f-46bcf37dfb6f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:28.521247 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7733c5b-4cd2-4055-970f-46bcf37dfb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:28.533092 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93f3ef-59fe-43c0-ad8d-d0ca3a01cdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:28.533951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93f3ef-59fe-43c0-ad8d-d0ca3a01cdb2 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:28.534282 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93f3ef-59fe-43c0-ad8d-d0ca3a01cdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:28.534598 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93f3ef-59fe-43c0-ad8d-d0ca3a01cdb2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:28.543712 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce93f3ef-59fe-43c0-ad8d-d0ca3a01cdb2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:28.544102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-c7733c5b-4cd2-4055-970f-46bcf37dfb6f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:28.545155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce93f3ef-59fe-43c0-ad8d-d0ca3a01cdb2 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:28.545758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-c7733c5b-4cd2-4055-970f-46bcf37dfb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:28.562155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-ce93f3ef-59fe-43c0-ad8d-d0ca3a01cdb2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:35:28.562155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:28.562155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:28.562155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:28.562155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:28.562155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:28.562155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:28.562155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:28.562155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:28.562155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:28.562155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:28.562155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:28.562155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:35:28.562155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:35:28.562155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:35:28.562155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:35:28.563162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:28.563162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:28.563162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:28.563162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:28.563162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:28.563162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:28.563162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:28.563162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:28.563162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:28.563162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:28.563162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:28.563162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:28.563162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:28.563162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:28.563162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:28.563162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:28.563162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:28.563162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:28.564396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:28.564396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:28.564396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:35:28.564396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:35:28.564396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:35:28.564396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:35:28.564396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:35:28.564396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:35:28.564396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:35:28.564396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:28.564396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 467/906] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jun 27 15:35:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:28.569009 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93f3ef-59fe-43c0-ad8d-d0ca3a01cdb2 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:28.645660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 440/907] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:35:28.660991 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7733c5b-4cd2-4055-970f-46bcf37dfb6f None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:28.675329 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:28.702809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93f3ef-59fe-43c0-ad8d-d0ca3a01cdb2 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:28.740764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ffd8c-e4fc-4e29-97cc-ee9cc0251d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:28.741786 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ffd8c-e4fc-4e29-97cc-ee9cc0251d77 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:28.742214 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ffd8c-e4fc-4e29-97cc-ee9cc0251d77 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:28.742636 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ffd8c-e4fc-4e29-97cc-ee9cc0251d77 None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:28.743727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:28.745029 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ffd8c-e4fc-4e29-97cc-ee9cc0251d77 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:28.745770 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ffd8c-e4fc-4e29-97cc-ee9cc0251d77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:28.747804 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ffd8c-e4fc-4e29-97cc-ee9cc0251d77 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:28.777751 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 468/908] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Jun 27 15:35:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:28.777751 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ffd8c-e4fc-4e29-97cc-ee9cc0251d77 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:28.804919 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7733c5b-4cd2-4055-970f-46bcf37dfb6f None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:28.847568 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4afae-cbca-4c4f-8886-7dd0ae37e93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:28.848442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4afae-cbca-4c4f-8886-7dd0ae37e93f None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:28.848734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4afae-cbca-4c4f-8886-7dd0ae37e93f None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:28.849013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4afae-cbca-4c4f-8886-7dd0ae37e93f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:28.849714 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4afae-cbca-4c4f-8886-7dd0ae37e93f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:28.851125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4afae-cbca-4c4f-8886-7dd0ae37e93f None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:28.864030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-2ff4afae-cbca-4c4f-8886-7dd0ae37e93f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 15:35:28.864030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:28.864030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:28.864030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:28.864030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:28.864030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:28.864030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:28.864030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:28.864030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 15:35:28.864030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:28.864030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:28.864030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:28.864030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:28.864030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:28.864030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:28.864030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:28.864030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:28.865050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:28.865050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:28.865050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:28.865050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:28.865050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:28.865050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:28.865050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:28.865050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:28.865050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:28.865050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:28.865050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:28.865050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:28.865050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:28.865050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:28.865050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:28.865050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:28.865949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:28.865949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:28.865949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:28.865949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:28.865949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:28.865949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:28.865949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:28.865949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:28.865949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:28.865949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:28.865949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:28.865949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:28.865949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 15:35:28.865949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:28.865949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 441/909] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:35:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:28.871747 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4afae-cbca-4c4f-8886-7dd0ae37e93f None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:28.919068 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ffd8c-e4fc-4e29-97cc-ee9cc0251d77 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:28.956952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550abcd-0dd8-4148-bd30-8fe5ea0d0a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:28.958015 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550abcd-0dd8-4148-bd30-8fe5ea0d0a39 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:28.958502 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550abcd-0dd8-4148-bd30-8fe5ea0d0a39 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:28.959008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550abcd-0dd8-4148-bd30-8fe5ea0d0a39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:28.968983 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:29.002690 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4afae-cbca-4c4f-8886-7dd0ae37e93f None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:29.025123 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:29.036807 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9550abcd-0dd8-4148-bd30-8fe5ea0d0a39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:29.038600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9550abcd-0dd8-4148-bd30-8fe5ea0d0a39 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:29.044213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4cb024-a517-4e8c-83a9-9f45a5f9a245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:29.044889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4cb024-a517-4e8c-83a9-9f45a5f9a245 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:29.045170 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4cb024-a517-4e8c-83a9-9f45a5f9a245 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:29.045442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4cb024-a517-4e8c-83a9-9f45a5f9a245 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:29.046294 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4cb024-a517-4e8c-83a9-9f45a5f9a245 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:29.046669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4cb024-a517-4e8c-83a9-9f45a5f9a245 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:29.048051 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4cb024-a517-4e8c-83a9-9f45a5f9a245 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:29.059266 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 442/910] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:35:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:29.060499 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4cb024-a517-4e8c-83a9-9f45a5f9a245 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:29.148304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 469/911] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:35:28 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:35:29.156006 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550abcd-0dd8-4148-bd30-8fe5ea0d0a39 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:29.174687 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4cb024-a517-4e8c-83a9-9f45a5f9a245 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:29.196576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:29.215304 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c19a0-5c68-4aed-be2f-2b972ffbfe7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:29.216207 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c19a0-5c68-4aed-be2f-2b972ffbfe7d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:29.216633 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c19a0-5c68-4aed-be2f-2b972ffbfe7d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:29.217164 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c19a0-5c68-4aed-be2f-2b972ffbfe7d None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:29.218152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c19a0-5c68-4aed-be2f-2b972ffbfe7d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:29.218599 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c19a0-5c68-4aed-be2f-2b972ffbfe7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:29.220316 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c19a0-5c68-4aed-be2f-2b972ffbfe7d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:29.231043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 443/912] 10.222.0.20 () {56 vars in 1114 bytes} [Mon Jun 27 15:35:29 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:29.232587 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c19a0-5c68-4aed-be2f-2b972ffbfe7d None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:29.265252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550abcd-0dd8-4148-bd30-8fe5ea0d0a39 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:29.288709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:29.313264 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef27783-e3bf-4145-98cd-2982f1933269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:29.314199 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef27783-e3bf-4145-98cd-2982f1933269 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:29.314560 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef27783-e3bf-4145-98cd-2982f1933269 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:29.314912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef27783-e3bf-4145-98cd-2982f1933269 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:29.326824 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef27783-e3bf-4145-98cd-2982f1933269 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:29.328318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef27783-e3bf-4145-98cd-2982f1933269 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:29.347870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-7ef27783-e3bf-4145-98cd-2982f1933269 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:29.347870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:29.347870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:29.347870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:29.347870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:29.347870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:29.347870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:29.347870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:29.347870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:29.347870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:29.347870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:29.347870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:29.347870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:29.347870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:29.347870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:29.349336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:29.349336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:29.349336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:29.349336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:29.349336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:29.349336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:29.349336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:29.349336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:29.349336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:29.349336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:29.349336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:29.349336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:29.349336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:29.349336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:29.349336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:29.349336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:29.349336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:29.350474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:29.350474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:29.350474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:29.350474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:29.356340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 470/913] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:29.356744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c19a0-5c68-4aed-be2f-2b972ffbfe7d None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:29.357468 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef27783-e3bf-4145-98cd-2982f1933269 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:29.396511 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3d975-60e8-4c49-8588-8a8e461a20f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:29.397546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3d975-60e8-4c49-8588-8a8e461a20f1 None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:29.398045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3d975-60e8-4c49-8588-8a8e461a20f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:29.398512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3d975-60e8-4c49-8588-8a8e461a20f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:29.399494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d3d975-60e8-4c49-8588-8a8e461a20f1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:29.401075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d3d975-60e8-4c49-8588-8a8e461a20f1 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:29.433922 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 444/914] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Jun 27 15:35:29 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:35:29.464873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:29.476997 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef27783-e3bf-4145-98cd-2982f1933269 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:29.532326 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc302a-751c-4ebb-800e-c5c4d3402064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:29.533071 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc302a-751c-4ebb-800e-c5c4d3402064 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:29.533361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc302a-751c-4ebb-800e-c5c4d3402064 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:29.533659 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc302a-751c-4ebb-800e-c5c4d3402064 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:29.534418 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc302a-751c-4ebb-800e-c5c4d3402064 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:29.534731 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc302a-751c-4ebb-800e-c5c4d3402064 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:29.536152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc302a-751c-4ebb-800e-c5c4d3402064 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:29.550891 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 471/915] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:29.557168 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3d975-60e8-4c49-8588-8a8e461a20f1 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:29.683768 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3d975-60e8-4c49-8588-8a8e461a20f1 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:29.686369 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:29.723801 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae652494-04a4-4dab-bdbd-52e92a8a61bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:29.724622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae652494-04a4-4dab-bdbd-52e92a8a61bd None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:29.724911 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae652494-04a4-4dab-bdbd-52e92a8a61bd None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:29.725188 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae652494-04a4-4dab-bdbd-52e92a8a61bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:29.734399 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae652494-04a4-4dab-bdbd-52e92a8a61bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:29.735882 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae652494-04a4-4dab-bdbd-52e92a8a61bd None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:29.752916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-ae652494-04a4-4dab-bdbd-52e92a8a61bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:29.752916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:29.752916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:29.752916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:29.752916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:29.752916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:29.752916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:29.752916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:29.752916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:29.752916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:29.752916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:29.752916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:29.752916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:29.752916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:29.752916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:29.754152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:29.754152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:29.754152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:29.754152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:29.754152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:29.754152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:29.754152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:29.754152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:29.754152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:29.754152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:29.754152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:29.754152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:29.754152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:29.754152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:29.754152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:29.754152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:29.754152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:29.755021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:29.755021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:29.755021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:29.755021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:29.755021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:29.755021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:29.755021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:29.755021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:29.757372 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 445/916] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:29.763523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc302a-751c-4ebb-800e-c5c4d3402064 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:29.880091 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc302a-751c-4ebb-800e-c5c4d3402064 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:29.900680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46217024-c5d3-447b-965f-c69da033fdb3 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:29.900999 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46217024-c5d3-447b-965f-c69da033fdb3 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:29.901242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46217024-c5d3-447b-965f-c69da033fdb3 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:29.903096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 446/917] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:29.908833 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451134b-ca0b-4642-9076-631d490e6da0 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:29.909115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451134b-ca0b-4642-9076-631d490e6da0 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:29.909389 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451134b-ca0b-4642-9076-631d490e6da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:29.922098 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055a32d-ea21-461a-a742-12b163f83ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:29.922773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055a32d-ea21-461a-a742-12b163f83ac7 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:29.923060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055a32d-ea21-461a-a742-12b163f83ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:29.923340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055a32d-ea21-461a-a742-12b163f83ac7 None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:29.924141 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055a32d-ea21-461a-a742-12b163f83ac7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:29.924454 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055a32d-ea21-461a-a742-12b163f83ac7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:29.925917 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055a32d-ea21-461a-a742-12b163f83ac7 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:29.928974 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-9451134b-ca0b-4642-9076-631d490e6da0 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:29.930211 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-9451134b-ca0b-4642-9076-631d490e6da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:29.941762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 472/918] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:29.947905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055a32d-ea21-461a-a742-12b163f83ac7 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:29.957281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:30.017721 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 447/919] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:35:30.035390 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451134b-ca0b-4642-9076-631d490e6da0 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:30.056602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055a32d-ea21-461a-a742-12b163f83ac7 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:30.093067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bac7d-86d7-4ea1-a8ba-2cbe2df9420c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:30.093775 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bac7d-86d7-4ea1-a8ba-2cbe2df9420c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:30.094067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bac7d-86d7-4ea1-a8ba-2cbe2df9420c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:30.094352 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bac7d-86d7-4ea1-a8ba-2cbe2df9420c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:30.095193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04bac7d-86d7-4ea1-a8ba-2cbe2df9420c None admin] RBAC: Adding query filter params (role.id=507fb7d64e4549c58ff859abf6160c43, scope.project.id=7be83fb6f20341139147373837320148, user.id=6655366e7eb84a26aa3f7fbe88af62b5) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:30.095510 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04bac7d-86d7-4ea1-a8ba-2cbe2df9420c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:30.097078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04bac7d-86d7-4ea1-a8ba-2cbe2df9420c None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:30.104201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 473/920] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:35:29 2022] GET /identity/v3/role_assignments?user.id=6655366e7eb84a26aa3f7fbe88af62b5&scope.project.id=7be83fb6f20341139147373837320148&role.id=507fb7d64e4549c58ff859abf6160c43 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:30.143364 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:30.153473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451134b-ca0b-4642-9076-631d490e6da0 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:30.194690 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765268d-6d12-4f65-bd8e-eac1fc5e0caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:30.195556 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765268d-6d12-4f65-bd8e-eac1fc5e0caf None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:30.195872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765268d-6d12-4f65-bd8e-eac1fc5e0caf None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:30.196259 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765268d-6d12-4f65-bd8e-eac1fc5e0caf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:30.196938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765268d-6d12-4f65-bd8e-eac1fc5e0caf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:30.198388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765268d-6d12-4f65-bd8e-eac1fc5e0caf None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:30.213122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-a765268d-6d12-4f65-bd8e-eac1fc5e0caf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 15:35:30.213122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:30.213122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:30.213122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:30.213122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:30.213122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:30.213122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:30.213122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:30.213122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 15:35:30.213122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:30.213122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:30.213122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:30.213122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:30.213122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:30.213122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:30.213122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:30.215377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:30.215377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:30.215377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:30.215377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:30.215377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:30.215377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:30.215377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:30.215377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:30.215377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:30.215377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:30.215377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:30.215377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:30.215377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:30.215377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:30.215377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:30.215377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:30.217333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:30.217333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:30.217333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:30.217333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:30.217333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:30.217333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:30.217333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:30.217333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:30.217333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:30.217333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:30.217333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:30.217333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:30.217333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:30.217333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 15:35:30.217333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:30.221706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 448/921] 10.222.0.20 () {56 vars in 1132 bytes} [Mon Jun 27 15:35:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:30.226846 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bac7d-86d7-4ea1-a8ba-2cbe2df9420c None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:30.314616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:30.352748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bac7d-86d7-4ea1-a8ba-2cbe2df9420c None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:30.390038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a88f83-4e2f-4d05-8aee-c1ac240f398d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:30.391029 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a88f83-4e2f-4d05-8aee-c1ac240f398d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:30.391495 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a88f83-4e2f-4d05-8aee-c1ac240f398d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:30.391961 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a88f83-4e2f-4d05-8aee-c1ac240f398d None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:30.392837 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a88f83-4e2f-4d05-8aee-c1ac240f398d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:30.393315 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a88f83-4e2f-4d05-8aee-c1ac240f398d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:30.394926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a88f83-4e2f-4d05-8aee-c1ac240f398d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:30.405128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 474/922] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jun 27 15:35:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:30.406731 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765268d-6d12-4f65-bd8e-eac1fc5e0caf None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:30.465234 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0841639-e74a-44bb-b842-afe2bd62f0d6 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:30.465562 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0841639-e74a-44bb-b842-afe2bd62f0d6 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:30.465819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0841639-e74a-44bb-b842-afe2bd62f0d6 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:30.467333 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 475/923] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:30.472773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81200a6-1169-47a9-8208-e553d4478ac7 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:30.473336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81200a6-1169-47a9-8208-e553d4478ac7 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:30.473876 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81200a6-1169-47a9-8208-e553d4478ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:30.494359 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-f81200a6-1169-47a9-8208-e553d4478ac7 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:30.495968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f81200a6-1169-47a9-8208-e553d4478ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:30.533978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765268d-6d12-4f65-bd8e-eac1fc5e0caf None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:30.572359 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92404357-0cab-45fc-82f8-4739445e5652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:30.573996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92404357-0cab-45fc-82f8-4739445e5652 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:30.574503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92404357-0cab-45fc-82f8-4739445e5652 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:30.575002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92404357-0cab-45fc-82f8-4739445e5652 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:30.575960 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92404357-0cab-45fc-82f8-4739445e5652 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:30.576473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92404357-0cab-45fc-82f8-4739445e5652 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:30.578123 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92404357-0cab-45fc-82f8-4739445e5652 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:30.588757 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 449/924] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jun 27 15:35:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:30.589520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 476/925] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:35:30.590363 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92404357-0cab-45fc-82f8-4739445e5652 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:30.620786 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81200a6-1169-47a9-8208-e553d4478ac7 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:30.625600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:30.690497 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:30.722968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92404357-0cab-45fc-82f8-4739445e5652 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:30.737953 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81200a6-1169-47a9-8208-e553d4478ac7 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:30.761699 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa827e-e045-42a5-8ef1-227135596782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:30.762610 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa827e-e045-42a5-8ef1-227135596782 None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:30.763042 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa827e-e045-42a5-8ef1-227135596782 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:30.763538 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa827e-e045-42a5-8ef1-227135596782 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:30.764523 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa827e-e045-42a5-8ef1-227135596782 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:30.766317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa827e-e045-42a5-8ef1-227135596782 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:30.775408 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7a05c-0b0d-4ab6-a855-71915f1e824e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:30.776767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7a05c-0b0d-4ab6-a855-71915f1e824e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:30.777098 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7a05c-0b0d-4ab6-a855-71915f1e824e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:30.777453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7a05c-0b0d-4ab6-a855-71915f1e824e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:30.778890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7a05c-0b0d-4ab6-a855-71915f1e824e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:30.780515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7a05c-0b0d-4ab6-a855-71915f1e824e None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:30.790673 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 450/926] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Jun 27 15:35:30 2022] POST /identity/v3/endpoints => generated 347 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:30.801715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-fad7a05c-0b0d-4ab6-a855-71915f1e824e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:35:30.801715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:30.801715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:30.801715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:30.801715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:30.801715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:30.801715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:30.801715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:30.801715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:35:30.801715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:30.801715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:30.801715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:30.801715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:30.801715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:30.801715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:30.801715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:30.801715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:30.802792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:30.802792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:30.802792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:30.802792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:30.802792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:30.802792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:30.802792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:30.802792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:30.802792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:30.802792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:30.802792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:30.802792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:30.802792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:30.802792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:30.802792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:30.802792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:30.803627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:30.803627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:30.803627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:30.803627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:30.803627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:30.803627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:30.803627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:30.803627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:30.803627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:30.803627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:30.803627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:30.803627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:30.803627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:35:30.803627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:30.803627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 477/927] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jun 27 15:35:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:30.806633 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa827e-e045-42a5-8ef1-227135596782 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:30.914121 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:30.949523 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa827e-e045-42a5-8ef1-227135596782 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:30.989456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8195cea-9496-4775-90d8-fdc1527a69ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:30.990176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195cea-9496-4775-90d8-fdc1527a69ca None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:30.990461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195cea-9496-4775-90d8-fdc1527a69ca None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:30.990735 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195cea-9496-4775-90d8-fdc1527a69ca None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:30.991552 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8195cea-9496-4775-90d8-fdc1527a69ca None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:30.991909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8195cea-9496-4775-90d8-fdc1527a69ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:30.994242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8195cea-9496-4775-90d8-fdc1527a69ca None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:31.010336 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 451/928] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jun 27 15:35:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:31.016562 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7a05c-0b0d-4ab6-a855-71915f1e824e None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:31.159898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7a05c-0b0d-4ab6-a855-71915f1e824e None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:31.188864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:31.199484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc503dc1-573a-4201-8d96-336c2777a8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:31.200539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc503dc1-573a-4201-8d96-336c2777a8ae None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:31.201029 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc503dc1-573a-4201-8d96-336c2777a8ae None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:31.201508 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc503dc1-573a-4201-8d96-336c2777a8ae None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:31.202456 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc503dc1-573a-4201-8d96-336c2777a8ae None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:31.202988 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc503dc1-573a-4201-8d96-336c2777a8ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:31.204596 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc503dc1-573a-4201-8d96-336c2777a8ae None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:31.211082 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 478/929] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jun 27 15:35:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:31.220150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8195cea-9496-4775-90d8-fdc1527a69ca None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:31.345238 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8195cea-9496-4775-90d8-fdc1527a69ca None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:31.384621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e190d9-c27f-4b99-a5b1-bd15cc2a4209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:31.385299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e190d9-c27f-4b99-a5b1-bd15cc2a4209 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:31.385585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e190d9-c27f-4b99-a5b1-bd15cc2a4209 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:31.385891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e190d9-c27f-4b99-a5b1-bd15cc2a4209 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:31.386692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e190d9-c27f-4b99-a5b1-bd15cc2a4209 None admin] RBAC: Adding query filter params (interface=public, service_id=f4c34302670f48fa9071fa0ec9ec26e9, region_id=RegionOne) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:31.387007 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e190d9-c27f-4b99-a5b1-bd15cc2a4209 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:31.388400 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e190d9-c27f-4b99-a5b1-bd15cc2a4209 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:31.394554 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 452/930] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Jun 27 15:35:31 2022] GET /identity/v3/endpoints?service_id=f4c34302670f48fa9071fa0ec9ec26e9&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:31.402940 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc503dc1-573a-4201-8d96-336c2777a8ae None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:31.516422 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc503dc1-573a-4201-8d96-336c2777a8ae None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:31.552652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75abf4-60da-46cc-a6f9-822b3bbcf269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:31.553337 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75abf4-60da-46cc-a6f9-822b3bbcf269 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:31.553624 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75abf4-60da-46cc-a6f9-822b3bbcf269 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:31.553943 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75abf4-60da-46cc-a6f9-822b3bbcf269 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:31.554701 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75abf4-60da-46cc-a6f9-822b3bbcf269 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:31.556805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75abf4-60da-46cc-a6f9-822b3bbcf269 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:31.563877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 479/931] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Jun 27 15:35:31 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:35:31.691463 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64de5d5-0e7e-400c-997c-a30ec661b35f None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:31.691751 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64de5d5-0e7e-400c-997c-a30ec661b35f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:31.692028 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64de5d5-0e7e-400c-997c-a30ec661b35f None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:31.693759 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 453/932] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:31.700260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00723bb-e049-4d5a-9011-db4d0e79c77a None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:31.700594 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00723bb-e049-4d5a-9011-db4d0e79c77a None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:31.700833 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00723bb-e049-4d5a-9011-db4d0e79c77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:31.720323 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-b00723bb-e049-4d5a-9011-db4d0e79c77a None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:31.721552 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-b00723bb-e049-4d5a-9011-db4d0e79c77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:31.779648 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:31.800860 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 480/933] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:31.847563 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64de5d5-0e7e-400c-997c-a30ec661b35f None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:31.852822 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:31.963701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64de5d5-0e7e-400c-997c-a30ec661b35f None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:32.000936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997216dc-5ed0-45c2-9773-1dcbe5e3cfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:32.001602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997216dc-5ed0-45c2-9773-1dcbe5e3cfef None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:32.001915 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997216dc-5ed0-45c2-9773-1dcbe5e3cfef None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:32.002192 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997216dc-5ed0-45c2-9773-1dcbe5e3cfef None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:32.002887 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997216dc-5ed0-45c2-9773-1dcbe5e3cfef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:32.004306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997216dc-5ed0-45c2-9773-1dcbe5e3cfef None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:32.005696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: WARNING keystone.server.flask.common [None req-997216dc-5ed0-45c2-9773-1dcbe5e3cfef None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 27 15:35:32.025925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.sql.core [None req-997216dc-5ed0-45c2-9773-1dcbe5e3cfef None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 27 15:35:32.025925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 27 15:35:32.025925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [parameters: {'id': '81079d17f2474395894fa3261776e10c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 27 15:35:32.025925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91941) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 27 15:35:32.033807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-997216dc-5ed0-45c2-9773-1dcbe5e3cfef None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 81079d17f2474395894fa3261776e10c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 81079d17f2474395894fa3261776e10c. Jun 27 15:35:32.033807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:32.033807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 15:35:32.033807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 15:35:32.033807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 15:35:32.033807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 15:35:32.033807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 15:35:32.033807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 15:35:32.033807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 15:35:32.033807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application conn.query(q) Jun 27 15:35:32.033807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 15:35:32.033807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 15:35:32.033807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 15:35:32.034589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application result.read() Jun 27 15:35:32.034589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 15:35:32.034589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 15:35:32.034589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 15:35:32.034589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 15:35:32.034589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 15:35:32.034589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 15:35:32.034589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 15:35:32.034589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 15:35:32.034589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 27 15:35:32.034589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:32.034589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 27 15:35:32.034589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:32.034589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:32.034589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 27 15:35:32.034589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 27 15:35:32.034589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 27 15:35:32.034589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 27 15:35:32.035524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 15:35:32.035524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application next(self.gen) Jun 27 15:35:32.035524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 27 15:35:32.035524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application yield resource Jun 27 15:35:32.035524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 15:35:32.035524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application next(self.gen) Jun 27 15:35:32.035524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 27 15:35:32.035524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self.session.rollback() Jun 27 15:35:32.035524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 27 15:35:32.035524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self.force_reraise() Jun 27 15:35:32.035524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 27 15:35:32.035524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise self.value Jun 27 15:35:32.035524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 27 15:35:32.035524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 27 15:35:32.035524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 27 15:35:32.035524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application session.commit() Jun 27 15:35:32.035524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 27 15:35:32.036405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 27 15:35:32.036405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 27 15:35:32.036405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._prepare_impl() Jun 27 15:35:32.036405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 27 15:35:32.036405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self.session.flush() Jun 27 15:35:32.036405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 27 15:35:32.036405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._flush(objects) Jun 27 15:35:32.036405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 27 15:35:32.036405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 27 15:35:32.036405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 27 15:35:32.036405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application compat.raise_( Jun 27 15:35:32.036405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 15:35:32.036405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception Jun 27 15:35:32.036405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 27 15:35:32.036405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application flush_context.execute() Jun 27 15:35:32.036405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 27 15:35:32.036405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rec.execute(self) Jun 27 15:35:32.037293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 27 15:35:32.037293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 27 15:35:32.037293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 27 15:35:32.037293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 27 15:35:32.037293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 27 15:35:32.037293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 27 15:35:32.037293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 27 15:35:32.037293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 27 15:35:32.037293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 27 15:35:32.037293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 27 15:35:32.037293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 27 15:35:32.037293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 27 15:35:32.037293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 27 15:35:32.037293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 27 15:35:32.037293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 27 15:35:32.038179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 27 15:35:32.038179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 15:35:32.038179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception Jun 27 15:35:32.038179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 15:35:32.038179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 15:35:32.038179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 15:35:32.038179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 15:35:32.038179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 15:35:32.038179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 15:35:32.038179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 15:35:32.038179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application conn.query(q) Jun 27 15:35:32.038179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 15:35:32.038179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 15:35:32.038179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 15:35:32.038179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application result.read() Jun 27 15:35:32.038179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 15:35:32.038179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 15:35:32.039045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 15:35:32.039045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 15:35:32.039045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 15:35:32.039045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 15:35:32.039045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 15:35:32.039045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 15:35:32.039045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 27 15:35:32.039045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 27 15:35:32.039045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [parameters: {'id': '81079d17f2474395894fa3261776e10c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 27 15:35:32.039045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 27 15:35:32.039045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:32.039045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:32.039045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:32.039045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:32.039045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 27 15:35:32.040148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 27 15:35:32.040148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 27 15:35:32.040148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 27 15:35:32.040148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 27 15:35:32.040148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:32.040148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:32.040148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:32.040148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:32.040148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:32.040148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:32.040148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:32.040148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:32.040148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:32.040148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:32.040148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:32.040148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:32.040148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:32.041060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:32.041060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 27 15:35:32.041060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 27 15:35:32.041060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:32.041060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:32.041060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 27 15:35:32.041060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 27 15:35:32.041060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 81079d17f2474395894fa3261776e10c. Jun 27 15:35:32.041060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:32.041060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 454/934] 10.222.0.20 () {60 vars in 1140 bytes} [Mon Jun 27 15:35:31 2022] POST /identity/v3/projects => generated 262 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:32.048883 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00723bb-e049-4d5a-9011-db4d0e79c77a None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:32.153955 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00723bb-e049-4d5a-9011-db4d0e79c77a None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:32.189768 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5d9a6-8188-4374-b8ec-1103d2ee25ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:32.190574 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5d9a6-8188-4374-b8ec-1103d2ee25ff None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:32.190854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5d9a6-8188-4374-b8ec-1103d2ee25ff None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:32.191127 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5d9a6-8188-4374-b8ec-1103d2ee25ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:32.198871 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5d9a6-8188-4374-b8ec-1103d2ee25ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:32.200317 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5d9a6-8188-4374-b8ec-1103d2ee25ff None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:32.215032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-52c5d9a6-8188-4374-b8ec-1103d2ee25ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:32.215032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:32.215032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:32.215032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:32.215032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:32.215032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:32.215032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:32.215032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:32.215032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:32.215032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:32.215032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:32.215032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:32.215032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:32.215032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:32.215032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:32.217045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:32.217045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:32.217045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:32.217045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:32.217045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:32.217045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:32.217045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:32.217045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:32.217045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:32.217045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:32.217045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:32.217045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:32.217045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:32.217045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:32.217045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:32.217045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:32.217045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:32.218305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:32.218305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:32.218305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:32.218305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:32.218305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:32.218305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:32.218305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:32.218305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:32.218305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 481/935] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:32.223966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997216dc-5ed0-45c2-9773-1dcbe5e3cfef None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:32.288655 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:32.305872 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66facf-abba-4853-bb95-e61b9c9e233b None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:32.306511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66facf-abba-4853-bb95-e61b9c9e233b None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:32.306997 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66facf-abba-4853-bb95-e61b9c9e233b None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:32.308900 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 482/936] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:32.315023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47670e7d-80dc-4364-8a38-da05b001064b None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:32.315809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47670e7d-80dc-4364-8a38-da05b001064b None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:32.316294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47670e7d-80dc-4364-8a38-da05b001064b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:32.334256 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-47670e7d-80dc-4364-8a38-da05b001064b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:32.335817 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-47670e7d-80dc-4364-8a38-da05b001064b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:32.341554 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997216dc-5ed0-45c2-9773-1dcbe5e3cfef None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:32.387581 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:32.391876 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5133af-c251-4f2b-869a-3f630758be2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:32.392656 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5133af-c251-4f2b-869a-3f630758be2b None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:32.393037 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5133af-c251-4f2b-869a-3f630758be2b None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:32.393321 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5133af-c251-4f2b-869a-3f630758be2b None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:32.394123 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5133af-c251-4f2b-869a-3f630758be2b None admin] RBAC: Adding query filter params (name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:32.394623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5133af-c251-4f2b-869a-3f630758be2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:32.397328 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5133af-c251-4f2b-869a-3f630758be2b None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:32.423993 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 455/937] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jun 27 15:35:32 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:32.424638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 483/938] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:35:32.443582 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5133af-c251-4f2b-869a-3f630758be2b None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:32.569927 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5133af-c251-4f2b-869a-3f630758be2b None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:32.598038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:32.613021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35461c8b-426c-4c65-b82c-90e45905f075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:32.614293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35461c8b-426c-4c65-b82c-90e45905f075 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:32.614776 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35461c8b-426c-4c65-b82c-90e45905f075 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:32.615242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35461c8b-426c-4c65-b82c-90e45905f075 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:32.626973 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35461c8b-426c-4c65-b82c-90e45905f075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:32.636098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-35461c8b-426c-4c65-b82c-90e45905f075 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:35:32.636098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:32.636098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:32.636098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:32.636098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:32.636098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:32.636098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:32.636098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:32.636098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:32.636098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:32.636098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:32.636098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:32.636098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:32.636098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:32.636098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:32.636098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:32.637048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:32.637048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:32.637048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:32.637048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:32.637048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:32.637048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:32.637048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:32.637048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:32.637048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:32.637048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:32.637048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:32.637048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:32.637048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:32.637048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:32.637048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:32.637048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:32.637890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:32.637890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:32.637890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:32.637890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:32.637890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:32.637890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:32.637890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:32.637890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:32.637890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:32.637890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:32.637890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:35:32.637890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:32.641704 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 456/939] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Jun 27 15:35:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:32.644865 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47670e7d-80dc-4364-8a38-da05b001064b None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:32.756256 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47670e7d-80dc-4364-8a38-da05b001064b None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:32.796216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd81a34-c1e6-4e13-b144-2c3b72dc6eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:32.796920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd81a34-c1e6-4e13-b144-2c3b72dc6eb2 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:32.797201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd81a34-c1e6-4e13-b144-2c3b72dc6eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:32.797475 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd81a34-c1e6-4e13-b144-2c3b72dc6eb2 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:32.798690 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd81a34-c1e6-4e13-b144-2c3b72dc6eb2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:32.798997 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd81a34-c1e6-4e13-b144-2c3b72dc6eb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:32.800437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd81a34-c1e6-4e13-b144-2c3b72dc6eb2 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:32.809664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 484/940] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Jun 27 15:35:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:32.815173 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35461c8b-426c-4c65-b82c-90e45905f075 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:32.922944 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35461c8b-426c-4c65-b82c-90e45905f075 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:32.966640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0c94d-8e39-4575-aa5c-bd649fe02464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:32.967814 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0c94d-8e39-4575-aa5c-bd649fe02464 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:32.968298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0c94d-8e39-4575-aa5c-bd649fe02464 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:32.968781 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0c94d-8e39-4575-aa5c-bd649fe02464 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:32.980469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0c94d-8e39-4575-aa5c-bd649fe02464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:32.982300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0c94d-8e39-4575-aa5c-bd649fe02464 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:33.000829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-19c0c94d-8e39-4575-aa5c-bd649fe02464 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:33.000829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:33.000829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:33.000829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:33.000829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:33.000829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:33.000829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:33.000829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:33.000829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:33.000829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:33.000829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:33.000829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:33.000829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:33.000829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:33.000829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:33.002036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:33.002036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:33.002036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:33.002036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:33.002036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:33.002036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:33.002036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:33.002036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:33.002036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:33.002036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:33.002036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:33.002036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:33.002036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:33.002036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:33.002036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:33.002036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:33.002036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:33.002937 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:33.002937 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:33.002937 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:33.002937 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:33.005004 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 457/941] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:33.013942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd81a34-c1e6-4e13-b144-2c3b72dc6eb2 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:33.052605 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:33.118418 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd81a34-c1e6-4e13-b144-2c3b72dc6eb2 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:33.120716 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f6377-495d-469a-8491-c3402cb0d3da None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:33.120997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f6377-495d-469a-8491-c3402cb0d3da None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:33.121274 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f6377-495d-469a-8491-c3402cb0d3da None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:33.122997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 458/942] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:33.128916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0c86e-7344-4323-be19-3d70228cba95 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:33.129472 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0c86e-7344-4323-be19-3d70228cba95 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:33.129956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0c86e-7344-4323-be19-3d70228cba95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:33.148662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-f9b0c86e-7344-4323-be19-3d70228cba95 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:33.151134 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f9b0c86e-7344-4323-be19-3d70228cba95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:33.166519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837b973-2d95-4d7e-a7db-19cbd46b5b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:33.167904 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837b973-2d95-4d7e-a7db-19cbd46b5b06 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:33.168575 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837b973-2d95-4d7e-a7db-19cbd46b5b06 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:33.168993 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837b973-2d95-4d7e-a7db-19cbd46b5b06 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:33.170155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837b973-2d95-4d7e-a7db-19cbd46b5b06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:33.170609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837b973-2d95-4d7e-a7db-19cbd46b5b06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:33.172462 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837b973-2d95-4d7e-a7db-19cbd46b5b06 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:33.178128 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:33.191426 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 485/943] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:33.201652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837b973-2d95-4d7e-a7db-19cbd46b5b06 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:33.231300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 459/944] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:35:33.253997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b0c86e-7344-4323-be19-3d70228cba95 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:33.323008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:33.329420 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837b973-2d95-4d7e-a7db-19cbd46b5b06 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:33.371931 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b0c86e-7344-4323-be19-3d70228cba95 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:33.375391 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ea89d-65f9-45ce-930f-086b5d89b14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:33.376220 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ea89d-65f9-45ce-930f-086b5d89b14e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:33.376502 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ea89d-65f9-45ce-930f-086b5d89b14e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:33.376776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ea89d-65f9-45ce-930f-086b5d89b14e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:33.387209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ea89d-65f9-45ce-930f-086b5d89b14e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:33.388943 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ea89d-65f9-45ce-930f-086b5d89b14e None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:33.410974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-d20ea89d-65f9-45ce-930f-086b5d89b14e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:35:33.410974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:33.410974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:33.410974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:33.410974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:33.410974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:33.410974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:33.410974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:33.410974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:33.410974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:33.410974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:33.410974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:33.410974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:35:33.410974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:35:33.410974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:35:33.410974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:35:33.411945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:33.411945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:33.411945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:33.411945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:33.411945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:33.411945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:33.411945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:33.411945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:33.411945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:33.411945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:33.411945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:33.411945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:33.411945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:33.411945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:33.411945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:33.411945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:33.411945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:33.411945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:33.412854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:33.412854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:33.412854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:35:33.412854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:35:33.412854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:35:33.412854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:35:33.412854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:35:33.412854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:35:33.412854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:35:33.412854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:33.412854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 486/945] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jun 27 15:35:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:33.418662 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ea89d-65f9-45ce-930f-086b5d89b14e None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:33.419593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b741f1e-6b5c-4702-9934-b3359a9a99fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:33.420685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b741f1e-6b5c-4702-9934-b3359a9a99fb None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:33.421199 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b741f1e-6b5c-4702-9934-b3359a9a99fb None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:33.421895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b741f1e-6b5c-4702-9934-b3359a9a99fb None admin] PATH_INFO: `/v3/services/compute` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:33.423730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b741f1e-6b5c-4702-9934-b3359a9a99fb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:33.425376 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b741f1e-6b5c-4702-9934-b3359a9a99fb None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:33.441488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-4b741f1e-6b5c-4702-9934-b3359a9a99fb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 15:35:33.441488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:33.441488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:33.441488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:33.441488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:33.441488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:33.441488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:33.441488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:33.441488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 15:35:33.441488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:33.441488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:33.441488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:33.441488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:33.441488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:33.441488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:33.441488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:33.441488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:33.442564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:33.442564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:33.442564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:33.442564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:33.442564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:33.442564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:33.442564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:33.442564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:33.442564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:33.442564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:33.442564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:33.442564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:33.442564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:33.442564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:33.442564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:33.442564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:33.443410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:33.443410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:33.443410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:33.443410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:33.443410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:33.443410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:33.443410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:33.443410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:33.443410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:33.443410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:33.443410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:33.443410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:33.443410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 15:35:33.443410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:33.448337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 460/946] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:33.454612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b741f1e-6b5c-4702-9934-b3359a9a99fb None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:33.482874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:33.536488 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ea89d-65f9-45ce-930f-086b5d89b14e None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:33.571498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b741f1e-6b5c-4702-9934-b3359a9a99fb None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:33.573628 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c31148-17d3-4fe9-bea1-b10d19a95d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:33.574606 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c31148-17d3-4fe9-bea1-b10d19a95d40 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:33.575095 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c31148-17d3-4fe9-bea1-b10d19a95d40 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:33.575580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c31148-17d3-4fe9-bea1-b10d19a95d40 None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:33.576626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c31148-17d3-4fe9-bea1-b10d19a95d40 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:33.577153 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c31148-17d3-4fe9-bea1-b10d19a95d40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:33.578816 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c31148-17d3-4fe9-bea1-b10d19a95d40 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:33.598215 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 487/947] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Jun 27 15:35:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:33.604093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c31148-17d3-4fe9-bea1-b10d19a95d40 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:33.615879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc51a7a-7424-481d-83aa-594518ef0d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:33.617367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc51a7a-7424-481d-83aa-594518ef0d4b None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:33.617966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc51a7a-7424-481d-83aa-594518ef0d4b None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:33.618448 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc51a7a-7424-481d-83aa-594518ef0d4b None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:33.619850 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc51a7a-7424-481d-83aa-594518ef0d4b None admin] RBAC: Adding query filter params (name=compute) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:33.620675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc51a7a-7424-481d-83aa-594518ef0d4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:33.622757 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc51a7a-7424-481d-83aa-594518ef0d4b None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:33.627254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:33.630880 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 461/948] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jun 27 15:35:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:33.639030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc51a7a-7424-481d-83aa-594518ef0d4b None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:33.723467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c31148-17d3-4fe9-bea1-b10d19a95d40 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:33.736427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:33.751495 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc51a7a-7424-481d-83aa-594518ef0d4b None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:33.764349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046843e-f16c-478f-ba8d-c1ca937105f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:33.765477 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046843e-f16c-478f-ba8d-c1ca937105f2 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:33.765985 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046843e-f16c-478f-ba8d-c1ca937105f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:33.766483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046843e-f16c-478f-ba8d-c1ca937105f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:33.778810 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046843e-f16c-478f-ba8d-c1ca937105f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:33.780700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046843e-f16c-478f-ba8d-c1ca937105f2 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:33.788028 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb102a-e892-432d-aaba-dfa967b3b9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:33.788703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb102a-e892-432d-aaba-dfa967b3b9ef None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:33.788988 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb102a-e892-432d-aaba-dfa967b3b9ef None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:33.789905 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb102a-e892-432d-aaba-dfa967b3b9ef None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:33.790329 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb102a-e892-432d-aaba-dfa967b3b9ef None admin] RBAC: Adding query filter params (type=compute) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:33.790636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb102a-e892-432d-aaba-dfa967b3b9ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:33.792017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb102a-e892-432d-aaba-dfa967b3b9ef None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:33.796894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 462/949] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jun 27 15:35:33 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:33.801487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-f046843e-f16c-478f-ba8d-c1ca937105f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:33.801487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:33.801487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:33.801487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:33.801487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:33.801487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:33.801487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:33.801487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:33.801487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:33.801487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:33.801487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:33.801487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:33.801487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:33.801487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:33.801487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:33.802609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:33.802609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:33.802609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:33.802609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:33.802609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:33.802609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:33.802609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:33.802609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:33.802609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:33.802609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:33.802609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:33.802609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:33.802609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:33.802609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:33.802609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:33.802609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:33.802609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:33.803489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:33.803489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:33.803489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:33.803489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:33.803871 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d2a66-6a53-4429-bf0f-3fe006678c68 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:33.804169 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d2a66-6a53-4429-bf0f-3fe006678c68 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:33.804457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d2a66-6a53-4429-bf0f-3fe006678c68 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:33.806834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 463/950] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:33.807546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 488/951] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:33.813321 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d2a66-6a53-4429-bf0f-3fe006678c68 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:33.815113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c215854-b6df-4ee8-a56a-adeac2870607 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:33.815610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c215854-b6df-4ee8-a56a-adeac2870607 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:33.816050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c215854-b6df-4ee8-a56a-adeac2870607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:33.838130 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-4c215854-b6df-4ee8-a56a-adeac2870607 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:33.839636 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-4c215854-b6df-4ee8-a56a-adeac2870607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:33.921140 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:33.937661 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 489/952] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:33.948456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d2a66-6a53-4429-bf0f-3fe006678c68 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:33.956757 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c215854-b6df-4ee8-a56a-adeac2870607 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:33.989494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d97636-6738-4f81-9dcf-190cca334760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:33.990196 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d97636-6738-4f81-9dcf-190cca334760 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:33.990481 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d97636-6738-4f81-9dcf-190cca334760 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:33.990757 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d97636-6738-4f81-9dcf-190cca334760 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:33.992175 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d97636-6738-4f81-9dcf-190cca334760 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:33.992175 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d97636-6738-4f81-9dcf-190cca334760 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:33.993655 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d97636-6738-4f81-9dcf-190cca334760 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:34.007580 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 464/953] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:34.013464 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d97636-6738-4f81-9dcf-190cca334760 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:34.031440 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:34.084718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c215854-b6df-4ee8-a56a-adeac2870607 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:34.128356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c6df3-1ee9-4a9a-90be-ce08c12d6c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:34.129552 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c6df3-1ee9-4a9a-90be-ce08c12d6c29 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:34.130073 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c6df3-1ee9-4a9a-90be-ce08c12d6c29 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:34.130647 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c6df3-1ee9-4a9a-90be-ce08c12d6c29 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:34.131712 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c6df3-1ee9-4a9a-90be-ce08c12d6c29 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:34.133926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c6df3-1ee9-4a9a-90be-ce08c12d6c29 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:34.142210 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d97636-6738-4f81-9dcf-190cca334760 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:34.147703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-4e2c6df3-1ee9-4a9a-90be-ce08c12d6c29 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:35:34.147703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:34.147703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:34.147703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:34.147703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:34.147703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:34.147703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:34.147703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:34.147703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:35:34.147703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:34.147703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:34.147703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:34.147703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:34.147703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:34.147703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:34.147703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:34.147703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:34.148976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:34.148976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:34.148976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:34.148976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:34.148976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:34.148976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:34.148976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:34.148976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:34.148976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:34.148976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:34.148976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:34.148976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:34.148976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:34.148976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:34.148976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:34.148976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:34.150513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:34.150513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:34.150513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:34.150513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:34.150513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:34.150513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:34.150513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:34.150513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:34.150513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:34.150513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:34.150513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:34.150513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:34.150513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:35:34.150513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:34.158411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 490/954] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jun 27 15:35:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:34.160000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c6df3-1ee9-4a9a-90be-ce08c12d6c29 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:34.181174 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e6aaf-eb30-48cd-a741-fa6d7cd9dcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:34.181700 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e6aaf-eb30-48cd-a741-fa6d7cd9dcdf None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:34.182039 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e6aaf-eb30-48cd-a741-fa6d7cd9dcdf None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:34.182318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e6aaf-eb30-48cd-a741-fa6d7cd9dcdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:34.190193 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e6aaf-eb30-48cd-a741-fa6d7cd9dcdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:34.191908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e6aaf-eb30-48cd-a741-fa6d7cd9dcdf None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:34.208081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-369e6aaf-eb30-48cd-a741-fa6d7cd9dcdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:34.208081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:34.208081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:34.208081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:34.208081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:34.208081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:34.208081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:34.208081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:34.208081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:34.208081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:34.208081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:34.208081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:34.208081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:34.208081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:34.208081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:34.209050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:34.209050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:34.209050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:34.209050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:34.209050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:34.209050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:34.209050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:34.209050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:34.209050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:34.209050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:34.209050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:34.209050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:34.209050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:34.209050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:34.209050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:34.209050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:34.209050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:34.210044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:34.210044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:34.210044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:34.210044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:34.210044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:34.210044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:34.210044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:34.210044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:34.213921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 465/955] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:34.215446 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e6aaf-eb30-48cd-a741-fa6d7cd9dcdf None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:34.239063 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:34.285933 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c6df3-1ee9-4a9a-90be-ce08c12d6c29 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:34.296187 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:34.329591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881d588-4dc4-4443-932f-0bf8648555e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:34.330781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881d588-4dc4-4443-932f-0bf8648555e7 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:34.331276 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e6aaf-eb30-48cd-a741-fa6d7cd9dcdf None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:34.331860 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881d588-4dc4-4443-932f-0bf8648555e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:34.332399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881d588-4dc4-4443-932f-0bf8648555e7 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:34.333405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881d588-4dc4-4443-932f-0bf8648555e7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:34.334010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881d588-4dc4-4443-932f-0bf8648555e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:34.335589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881d588-4dc4-4443-932f-0bf8648555e7 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:34.342593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 491/956] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jun 27 15:35:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:34.349248 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881d588-4dc4-4443-932f-0bf8648555e7 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:34.377527 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7bc78-7f7a-4dbc-a5a6-9deb2d31fa3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:34.378576 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7bc78-7f7a-4dbc-a5a6-9deb2d31fa3c None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:34.379349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7bc78-7f7a-4dbc-a5a6-9deb2d31fa3c None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:34.379812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7bc78-7f7a-4dbc-a5a6-9deb2d31fa3c None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:34.381478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c7bc78-7f7a-4dbc-a5a6-9deb2d31fa3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:34.381971 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c7bc78-7f7a-4dbc-a5a6-9deb2d31fa3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:34.385388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c7bc78-7f7a-4dbc-a5a6-9deb2d31fa3c None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:34.403469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 466/957] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:34.409469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7bc78-7f7a-4dbc-a5a6-9deb2d31fa3c None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:34.469569 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881d588-4dc4-4443-932f-0bf8648555e7 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:34.499964 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:34.510170 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9529a-acea-443c-a447-e5b753555ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:34.511037 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9529a-acea-443c-a447-e5b753555ef1 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:34.511396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9529a-acea-443c-a447-e5b753555ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:34.511751 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9529a-acea-443c-a447-e5b753555ef1 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:34.512796 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9529a-acea-443c-a447-e5b753555ef1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:34.513015 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9529a-acea-443c-a447-e5b753555ef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:34.514476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9529a-acea-443c-a447-e5b753555ef1 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:34.525492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 492/958] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jun 27 15:35:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:34.526503 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9529a-acea-443c-a447-e5b753555ef1 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:34.548694 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7bc78-7f7a-4dbc-a5a6-9deb2d31fa3c None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:34.580949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:34.590699 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d4c74-9ba0-4fe3-b212-b457c2f5849e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:34.592134 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d4c74-9ba0-4fe3-b212-b457c2f5849e None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:34.592592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d4c74-9ba0-4fe3-b212-b457c2f5849e None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:34.593258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d4c74-9ba0-4fe3-b212-b457c2f5849e None admin] PATH_INFO: `/v3/projects/7be83fb6f20341139147373837320148/users/6655366e7eb84a26aa3f7fbe88af62b5/roles/507fb7d64e4549c58ff859abf6160c43` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:34.596670 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d4c74-9ba0-4fe3-b212-b457c2f5849e None admin] RBAC: Authorizing `identity:create_grant(project_id=7be83fb6f20341139147373837320148, user_id=6655366e7eb84a26aa3f7fbe88af62b5, role_id=507fb7d64e4549c58ff859abf6160c43)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:34.598701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d4c74-9ba0-4fe3-b212-b457c2f5849e None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:34.621200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 467/959] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:35:34 2022] PUT /identity/v3/projects/7be83fb6f20341139147373837320148/users/6655366e7eb84a26aa3f7fbe88af62b5/roles/507fb7d64e4549c58ff859abf6160c43 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:35:34.664819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9529a-acea-443c-a447-e5b753555ef1 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:34.694915 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e142b7-ec5d-4496-b6d3-4d82c586617f None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:34.695569 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e142b7-ec5d-4496-b6d3-4d82c586617f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:34.696958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e142b7-ec5d-4496-b6d3-4d82c586617f None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:34.699952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 468/960] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:34.705916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31eb3b0-63e3-4143-a4f8-91740c3b47ef None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:34.706220 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31eb3b0-63e3-4143-a4f8-91740c3b47ef None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:34.706475 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31eb3b0-63e3-4143-a4f8-91740c3b47ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:34.713912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840c725-b598-4b51-978b-c82c2652d0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:34.714908 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840c725-b598-4b51-978b-c82c2652d0a1 None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:34.715355 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840c725-b598-4b51-978b-c82c2652d0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:34.715779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840c725-b598-4b51-978b-c82c2652d0a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:34.716681 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3840c725-b598-4b51-978b-c82c2652d0a1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:34.718262 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3840c725-b598-4b51-978b-c82c2652d0a1 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:34.726573 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-a31eb3b0-63e3-4143-a4f8-91740c3b47ef None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:34.728267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a31eb3b0-63e3-4143-a4f8-91740c3b47ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:34.742491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 493/961] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Jun 27 15:35:34 2022] POST /identity/v3/endpoints => generated 349 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:34.839005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 469/962] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jun 27 15:35:34.864742 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840c725-b598-4b51-978b-c82c2652d0a1 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:34.920056 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:34.986965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840c725-b598-4b51-978b-c82c2652d0a1 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:35.027458 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f9df2-3398-40aa-8e8b-3a6458275c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:35.028138 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f9df2-3398-40aa-8e8b-3a6458275c5c None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:35.028424 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f9df2-3398-40aa-8e8b-3a6458275c5c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:35.028697 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f9df2-3398-40aa-8e8b-3a6458275c5c None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:35.029412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f9df2-3398-40aa-8e8b-3a6458275c5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:35.030937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f9df2-3398-40aa-8e8b-3a6458275c5c None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:35.032373 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: WARNING keystone.server.flask.common [None req-3a0f9df2-3398-40aa-8e8b-3a6458275c5c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 27 15:35:35.045733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.sql.core [None req-3a0f9df2-3398-40aa-8e8b-3a6458275c5c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 27 15:35:35.045733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 27 15:35:35.045733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [parameters: {'user_id': '3c303e1dd0574b25b9b6d9d269ba5d04', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 27 15:35:35.045733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91942) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 27 15:35:35.063604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-3a0f9df2-3398-40aa-8e8b-3a6458275c5c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 27 15:35:35.063604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:35.063604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 15:35:35.063604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 15:35:35.063604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 15:35:35.063604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 15:35:35.063604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 15:35:35.063604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 15:35:35.063604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 15:35:35.063604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application conn.query(q) Jun 27 15:35:35.063604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 15:35:35.063604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 15:35:35.063604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 15:35:35.063604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application result.read() Jun 27 15:35:35.063604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 15:35:35.064781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 15:35:35.064781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 15:35:35.064781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 15:35:35.064781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 15:35:35.064781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 15:35:35.064781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 15:35:35.064781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 15:35:35.064781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 27 15:35:35.064781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:35.064781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 27 15:35:35.064781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:35.064781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:35.064781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 27 15:35:35.064781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 27 15:35:35.064781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 27 15:35:35.064781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 27 15:35:35.064781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 15:35:35.064781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application next(self.gen) Jun 27 15:35:35.066178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 27 15:35:35.066178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application yield resource Jun 27 15:35:35.066178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 15:35:35.066178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application next(self.gen) Jun 27 15:35:35.066178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 27 15:35:35.066178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self.session.rollback() Jun 27 15:35:35.066178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 27 15:35:35.066178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self.force_reraise() Jun 27 15:35:35.066178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 27 15:35:35.066178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise self.value Jun 27 15:35:35.066178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 27 15:35:35.066178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 27 15:35:35.066178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 27 15:35:35.066178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application session.commit() Jun 27 15:35:35.066178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 27 15:35:35.066178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 27 15:35:35.066178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 27 15:35:35.067319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._prepare_impl() Jun 27 15:35:35.067319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 27 15:35:35.067319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self.session.flush() Jun 27 15:35:35.067319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 27 15:35:35.067319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._flush(objects) Jun 27 15:35:35.067319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 27 15:35:35.067319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 27 15:35:35.067319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 27 15:35:35.067319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application compat.raise_( Jun 27 15:35:35.067319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 15:35:35.067319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception Jun 27 15:35:35.067319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 27 15:35:35.067319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application flush_context.execute() Jun 27 15:35:35.067319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 27 15:35:35.067319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rec.execute(self) Jun 27 15:35:35.067319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 27 15:35:35.067319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 27 15:35:35.068218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 27 15:35:35.068218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 27 15:35:35.068218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 27 15:35:35.068218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 27 15:35:35.068218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 27 15:35:35.068218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 27 15:35:35.068218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 27 15:35:35.068218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 27 15:35:35.068218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 27 15:35:35.068218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 27 15:35:35.068218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 27 15:35:35.068218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 27 15:35:35.068218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 27 15:35:35.068218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 27 15:35:35.068218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 15:35:35.068218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception Jun 27 15:35:35.069066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 15:35:35.069066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 15:35:35.069066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 15:35:35.069066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 15:35:35.069066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 15:35:35.069066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 15:35:35.069066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 15:35:35.069066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application conn.query(q) Jun 27 15:35:35.069066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 15:35:35.069066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 15:35:35.069066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 15:35:35.069066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application result.read() Jun 27 15:35:35.069066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 15:35:35.069066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 15:35:35.069066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 15:35:35.069066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 15:35:35.074382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 15:35:35.074382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 15:35:35.074382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 15:35:35.074382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 15:35:35.074382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 27 15:35:35.074382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 27 15:35:35.074382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [parameters: {'user_id': '3c303e1dd0574b25b9b6d9d269ba5d04', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 27 15:35:35.074382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 27 15:35:35.074382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:35.074382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:35.074382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:35.074382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:35.074382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:35.074382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:35.074382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:35.076284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:35.076284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:35.076284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:35.076284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:35.076284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:35.076284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:35.076284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:35.076284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 27 15:35:35.076284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 27 15:35:35.076284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:35.076284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:35.076284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:35.076284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:35.076284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:35.076284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:35.076284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 27 15:35:35.076284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 27 15:35:35.078499 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 27 15:35:35.078499 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 27 15:35:35.078499 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 27 15:35:35.078499 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 27 15:35:35.078499 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 27 15:35:35.078499 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:35.078499 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 494/963] 10.222.0.20 () {60 vars in 1131 bytes} [Mon Jun 27 15:35:34 2022] POST /identity/v3/users => generated 160 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:35.079434 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31eb3b0-63e3-4143-a4f8-91740c3b47ef None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:35.184972 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31eb3b0-63e3-4143-a4f8-91740c3b47ef None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:35.203651 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:35.221318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92deab4e-0add-44fd-b118-ed415f7f144c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:35.222205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92deab4e-0add-44fd-b118-ed415f7f144c None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:35.222487 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92deab4e-0add-44fd-b118-ed415f7f144c None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:35.222810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92deab4e-0add-44fd-b118-ed415f7f144c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:35.230457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92deab4e-0add-44fd-b118-ed415f7f144c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:35.231906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92deab4e-0add-44fd-b118-ed415f7f144c None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:35.247403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-92deab4e-0add-44fd-b118-ed415f7f144c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 15:35:35.247403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:35.247403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:35.247403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:35.247403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:35.247403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:35.247403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:35.247403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:35.247403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:35.247403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:35.247403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:35.247403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:35.247403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:35:35.247403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:35:35.247403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:35:35.247403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:35:35.249657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:35.249657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:35.249657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:35.249657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:35.249657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:35.249657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:35.249657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:35.249657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:35.249657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:35.249657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:35.249657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:35.249657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:35.249657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:35.249657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:35.249657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:35.249657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:35.249657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:35.249657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:35.251979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:35.251979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:35.251979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:35:35.251979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:35:35.251979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:35:35.251979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:35:35.251979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:35:35.251979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:35:35.251979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 15:35:35.251979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:35.251979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 470/964] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:35:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:35.257671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f9df2-3398-40aa-8e8b-3a6458275c5c None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:35.364765 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f9df2-3398-40aa-8e8b-3a6458275c5c None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:35.375494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:35.402882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e6514-b8e3-48f4-b03e-ab4e62dd75eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:35.403300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e6514-b8e3-48f4-b03e-ab4e62dd75eb None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:35.403466 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e6514-b8e3-48f4-b03e-ab4e62dd75eb None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:35.403728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e6514-b8e3-48f4-b03e-ab4e62dd75eb None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:35.404660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e6514-b8e3-48f4-b03e-ab4e62dd75eb None admin] RBAC: Adding query filter params (name=cinder) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:35.405333 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e6514-b8e3-48f4-b03e-ab4e62dd75eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:35.407471 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e6514-b8e3-48f4-b03e-ab4e62dd75eb None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:35.448967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 495/965] 10.222.0.20 () {56 vars in 1101 bytes} [Mon Jun 27 15:35:35 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:35.978772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701637e5-70c9-4a11-8e38-aaad8f305c8c None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:35.979278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701637e5-70c9-4a11-8e38-aaad8f305c8c None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:35.979355 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701637e5-70c9-4a11-8e38-aaad8f305c8c None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:35.980723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 471/966] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:35.985929 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2eb4cf-28a9-44c3-b306-389ce78c8f8d None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:35.986339 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2eb4cf-28a9-44c3-b306-389ce78c8f8d None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:35.986496 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2eb4cf-28a9-44c3-b306-389ce78c8f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:36.004980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-1e2eb4cf-28a9-44c3-b306-389ce78c8f8d None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:36.006605 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1e2eb4cf-28a9-44c3-b306-389ce78c8f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:36.081670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 496/967] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:36.102485 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701637e5-70c9-4a11-8e38-aaad8f305c8c None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:36.208088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701637e5-70c9-4a11-8e38-aaad8f305c8c None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:36.241244 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b838f6c-7455-41b5-b712-688abacd0c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:36.241931 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b838f6c-7455-41b5-b712-688abacd0c33 None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:36.242217 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b838f6c-7455-41b5-b712-688abacd0c33 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:36.242495 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b838f6c-7455-41b5-b712-688abacd0c33 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:36.243860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b838f6c-7455-41b5-b712-688abacd0c33 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:36.245223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b838f6c-7455-41b5-b712-688abacd0c33 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:36.267268 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 472/968] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jun 27 15:35:36 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:36.758550 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2412e-f804-4d2c-9823-534cdb92e5e6 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:36.759036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2412e-f804-4d2c-9823-534cdb92e5e6 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:36.759294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2412e-f804-4d2c-9823-534cdb92e5e6 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:36.760670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 497/969] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:36.766550 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7dfbb-0a85-4358-9fc2-3258e09fbd8e None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:36.766845 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7dfbb-0a85-4358-9fc2-3258e09fbd8e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:36.767251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7dfbb-0a85-4358-9fc2-3258e09fbd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:36.791218 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-1fd7dfbb-0a85-4358-9fc2-3258e09fbd8e None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:36.792466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1fd7dfbb-0a85-4358-9fc2-3258e09fbd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:36.830447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:36.884328 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 473/970] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:36.907444 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2412e-f804-4d2c-9823-534cdb92e5e6 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:36.980989 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f32b8-48b8-4d3a-907b-2d3371ba2c83 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:36.981527 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f32b8-48b8-4d3a-907b-2d3371ba2c83 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:36.981995 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f32b8-48b8-4d3a-907b-2d3371ba2c83 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:36.984120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 474/971] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:36.989481 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e48222-cf97-40e2-8a54-3991520af6c7 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:36.989807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e48222-cf97-40e2-8a54-3991520af6c7 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:36.990094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e48222-cf97-40e2-8a54-3991520af6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:37.010665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-93e48222-cf97-40e2-8a54-3991520af6c7 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:37.011794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-93e48222-cf97-40e2-8a54-3991520af6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:37.023298 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:37.032114 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2412e-f804-4d2c-9823-534cdb92e5e6 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:37.068390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985f6e2-01e3-4f4d-bb55-2937499d0c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:37.069494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985f6e2-01e3-4f4d-bb55-2937499d0c8c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:37.069982 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985f6e2-01e3-4f4d-bb55-2937499d0c8c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:37.070476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985f6e2-01e3-4f4d-bb55-2937499d0c8c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:37.082261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5985f6e2-01e3-4f4d-bb55-2937499d0c8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:37.083469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 475/972] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:35:37.091201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-5985f6e2-01e3-4f4d-bb55-2937499d0c8c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:35:37.091201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:37.091201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:37.091201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:37.091201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:37.091201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:37.091201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:37.091201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:37.091201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:37.091201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:37.091201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:37.091201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:37.091201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:37.091201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:37.091201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:37.091201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:37.092254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:37.092254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:37.092254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:37.092254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:37.092254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:37.092254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:37.092254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:37.092254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:37.092254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:37.092254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:37.092254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:37.092254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:37.092254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:37.092254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:37.092254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:37.092254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:37.093061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:37.093061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:37.093061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:37.093061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:37.093061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:37.093061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:37.093061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:37.093061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:37.093061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:37.093061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:37.093061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:35:37.093061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:37.097172 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 498/973] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Jun 27 15:35:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:37.098374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e48222-cf97-40e2-8a54-3991520af6c7 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:37.108085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985f6e2-01e3-4f4d-bb55-2937499d0c8c None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:37.204617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:37.212778 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e48222-cf97-40e2-8a54-3991520af6c7 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:37.224947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985f6e2-01e3-4f4d-bb55-2937499d0c8c None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:37.248042 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2fc04-11d8-4a06-af05-b5ce7683d06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:37.248715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2fc04-11d8-4a06-af05-b5ce7683d06b None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:37.249002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2fc04-11d8-4a06-af05-b5ce7683d06b None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:37.249279 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2fc04-11d8-4a06-af05-b5ce7683d06b None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:37.250166 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2fc04-11d8-4a06-af05-b5ce7683d06b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:37.250477 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2fc04-11d8-4a06-af05-b5ce7683d06b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:37.251859 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2fc04-11d8-4a06-af05-b5ce7683d06b None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:37.260811 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 476/974] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Jun 27 15:35:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:37.263069 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f78ed-f3d3-46a7-a0d7-aab5af048a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:37.264042 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f78ed-f3d3-46a7-a0d7-aab5af048a26 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:37.264544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f78ed-f3d3-46a7-a0d7-aab5af048a26 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:37.264971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f78ed-f3d3-46a7-a0d7-aab5af048a26 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:37.265821 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f78ed-f3d3-46a7-a0d7-aab5af048a26 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:37.266386 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2fc04-11d8-4a06-af05-b5ce7683d06b None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:37.267504 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f78ed-f3d3-46a7-a0d7-aab5af048a26 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:37.281053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-641f78ed-f3d3-46a7-a0d7-aab5af048a26 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:35:37.281053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:37.281053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:37.281053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:37.281053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:37.281053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:37.281053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:37.281053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:37.281053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:35:37.281053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:37.281053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:37.281053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:37.281053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:37.281053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:37.281053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:37.281053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:37.281053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:37.282398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:37.282398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:37.282398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:37.282398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:37.282398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:37.282398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:37.282398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:37.282398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:37.282398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:37.282398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:37.282398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:37.282398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:37.282398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:37.282398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:37.282398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:37.282398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:37.283436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:37.283436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:37.283436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:37.283436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:37.283436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:37.283436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:37.283436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:37.283436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:37.283436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:37.283436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:37.283436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:37.283436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:37.283436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:35:37.283436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:37.290885 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 499/975] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jun 27 15:35:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:37.291867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f78ed-f3d3-46a7-a0d7-aab5af048a26 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:37.314041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:37.382090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2fc04-11d8-4a06-af05-b5ce7683d06b None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:37.409041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f78ed-f3d3-46a7-a0d7-aab5af048a26 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:37.418996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02f821-bce4-46df-92eb-41f35003a415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:37.419802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02f821-bce4-46df-92eb-41f35003a415 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:37.420084 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02f821-bce4-46df-92eb-41f35003a415 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:37.420355 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02f821-bce4-46df-92eb-41f35003a415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:37.431451 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02f821-bce4-46df-92eb-41f35003a415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:37.432902 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02f821-bce4-46df-92eb-41f35003a415 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:37.448449 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528a7b7-af66-485e-9287-87a64bd87bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:37.449118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528a7b7-af66-485e-9287-87a64bd87bda None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:37.449401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528a7b7-af66-485e-9287-87a64bd87bda None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:37.449700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528a7b7-af66-485e-9287-87a64bd87bda None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:37.450509 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528a7b7-af66-485e-9287-87a64bd87bda None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:37.450824 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528a7b7-af66-485e-9287-87a64bd87bda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:37.450938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-1a02f821-bce4-46df-92eb-41f35003a415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:37.450938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:37.450938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:37.450938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:37.450938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:37.450938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:37.450938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:37.450938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:37.450938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:37.450938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:37.450938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:37.450938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:37.450938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:37.450938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:37.450938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:37.451772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:37.451772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:37.451772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:37.451772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:37.451772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:37.451772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:37.451772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:37.451772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:37.451772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:37.451772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:37.451772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:37.451772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:37.451772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:37.451772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:37.451772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:37.451772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:37.451772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:37.457832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:37.457832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:37.457832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:37.457832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:37.458093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528a7b7-af66-485e-9287-87a64bd87bda None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:37.458188 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 477/976] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:37.458188 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02f821-bce4-46df-92eb-41f35003a415 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:37.463052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 500/977] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jun 27 15:35:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:37.468404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528a7b7-af66-485e-9287-87a64bd87bda None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:37.474969 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:37.558741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:37.574847 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02f821-bce4-46df-92eb-41f35003a415 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:37.577337 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528a7b7-af66-485e-9287-87a64bd87bda None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:37.612748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe12d3-c16c-4804-b32e-b44849f2d598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:37.613489 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe12d3-c16c-4804-b32e-b44849f2d598 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:37.613802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe12d3-c16c-4804-b32e-b44849f2d598 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:37.614077 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe12d3-c16c-4804-b32e-b44849f2d598 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:37.614796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe12d3-c16c-4804-b32e-b44849f2d598 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:37.615102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe12d3-c16c-4804-b32e-b44849f2d598 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:37.615936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5536f599-db32-4a0c-a0c3-2d43cf2dbc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:37.616469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe12d3-c16c-4804-b32e-b44849f2d598 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:37.616636 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536f599-db32-4a0c-a0c3-2d43cf2dbc13 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:37.616885 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536f599-db32-4a0c-a0c3-2d43cf2dbc13 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:37.617165 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536f599-db32-4a0c-a0c3-2d43cf2dbc13 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:37.617952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536f599-db32-4a0c-a0c3-2d43cf2dbc13 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:37.618263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536f599-db32-4a0c-a0c3-2d43cf2dbc13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:37.619617 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536f599-db32-4a0c-a0c3-2d43cf2dbc13 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:37.624632 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 501/978] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jun 27 15:35:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:37.630238 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5536f599-db32-4a0c-a0c3-2d43cf2dbc13 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:37.632733 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 478/979] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:37.638335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe12d3-c16c-4804-b32e-b44849f2d598 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:37.706860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:37.738267 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5536f599-db32-4a0c-a0c3-2d43cf2dbc13 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:37.744465 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe12d3-c16c-4804-b32e-b44849f2d598 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:37.774644 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad9341-209a-468a-b2cc-88cdc6a864c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:37.775320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad9341-209a-468a-b2cc-88cdc6a864c6 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:37.775616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad9341-209a-468a-b2cc-88cdc6a864c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:37.775870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad9341-209a-468a-b2cc-88cdc6a864c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:37.776665 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad9341-209a-468a-b2cc-88cdc6a864c6 None admin] RBAC: Adding query filter params (interface=admin, service_id=f4c34302670f48fa9071fa0ec9ec26e9, region_id=RegionOne) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:37.776979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad9341-209a-468a-b2cc-88cdc6a864c6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:37.778357 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad9341-209a-468a-b2cc-88cdc6a864c6 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:37.781201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeca49a-16f0-44f6-9b05-b590f8b23b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:37.782083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeca49a-16f0-44f6-9b05-b590f8b23b18 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:37.782322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeca49a-16f0-44f6-9b05-b590f8b23b18 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:37.782596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeca49a-16f0-44f6-9b05-b590f8b23b18 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:37.783799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 502/980] 10.222.0.20 () {56 vars in 1249 bytes} [Mon Jun 27 15:35:37 2022] GET /identity/v3/endpoints?service_id=f4c34302670f48fa9071fa0ec9ec26e9&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:37.788856 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad9341-209a-468a-b2cc-88cdc6a864c6 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:37.794294 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeca49a-16f0-44f6-9b05-b590f8b23b18 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:37.795740 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeca49a-16f0-44f6-9b05-b590f8b23b18 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:37.807383 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:37.813692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-3eeca49a-16f0-44f6-9b05-b590f8b23b18 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:35:37.813692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:37.813692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:37.813692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:37.813692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:37.813692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:37.813692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:37.813692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:37.813692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:37.813692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:37.813692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:37.813692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:37.813692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:35:37.813692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:35:37.813692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:35:37.813692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:35:37.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:37.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:37.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:37.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:37.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:37.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:37.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:37.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:37.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:37.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:37.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:37.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:37.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:37.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:37.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:37.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:37.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:37.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:37.815617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:37.815617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:37.815617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:35:37.815617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:35:37.815617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:35:37.815617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:35:37.815617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:35:37.815617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:35:37.815617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 15:35:37.815617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:37.815617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 479/981] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jun 27 15:35:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:37.821726 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeca49a-16f0-44f6-9b05-b590f8b23b18 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:37.912797 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad9341-209a-468a-b2cc-88cdc6a864c6 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:37.933748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeca49a-16f0-44f6-9b05-b590f8b23b18 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:37.948686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771de256-6b42-4d69-b60a-03e9a56f38df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:37.949357 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771de256-6b42-4d69-b60a-03e9a56f38df None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:37.949662 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771de256-6b42-4d69-b60a-03e9a56f38df None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:37.949944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771de256-6b42-4d69-b60a-03e9a56f38df None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:37.950623 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771de256-6b42-4d69-b60a-03e9a56f38df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:37.952076 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771de256-6b42-4d69-b60a-03e9a56f38df None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:37.958434 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 503/982] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Jun 27 15:35:37 2022] GET /identity/v3/services => generated 1727 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:35:37.964952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:37.974595 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6eadb1-63cf-4dc0-9bc4-e2b368f51680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:37.975289 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6eadb1-63cf-4dc0-9bc4-e2b368f51680 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:37.975569 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6eadb1-63cf-4dc0-9bc4-e2b368f51680 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:37.975841 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6eadb1-63cf-4dc0-9bc4-e2b368f51680 None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:37.976713 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6eadb1-63cf-4dc0-9bc4-e2b368f51680 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:37.977571 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6eadb1-63cf-4dc0-9bc4-e2b368f51680 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:37.979084 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6eadb1-63cf-4dc0-9bc4-e2b368f51680 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:38.000528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 480/983] 10.222.0.20 () {56 vars in 1149 bytes} [Mon Jun 27 15:35:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:38.005983 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771de256-6b42-4d69-b60a-03e9a56f38df None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:38.091141 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:38.116699 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771de256-6b42-4d69-b60a-03e9a56f38df None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:38.152964 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051fd62-2b8f-485f-a2c8-3c779960cd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:38.153614 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051fd62-2b8f-485f-a2c8-3c779960cd88 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:38.153956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051fd62-2b8f-485f-a2c8-3c779960cd88 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:38.154231 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051fd62-2b8f-485f-a2c8-3c779960cd88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:38.221904 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051fd62-2b8f-485f-a2c8-3c779960cd88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:38.223341 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051fd62-2b8f-485f-a2c8-3c779960cd88 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:38.298379 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:38.328360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 504/984] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:35:38 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 15:35:38.330090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6eadb1-63cf-4dc0-9bc4-e2b368f51680 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:38.369038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bbb7e-78d4-4dab-ae98-77b99749d864 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:38.369253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bbb7e-78d4-4dab-ae98-77b99749d864 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:38.369534 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bbb7e-78d4-4dab-ae98-77b99749d864 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:38.370958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 505/985] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:38.376542 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686257b-bade-40d5-9ba0-bcf2d8458644 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:38.376840 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686257b-bade-40d5-9ba0-bcf2d8458644 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:38.377121 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686257b-bade-40d5-9ba0-bcf2d8458644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:38.395275 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-e686257b-bade-40d5-9ba0-bcf2d8458644 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:38.396602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e686257b-bade-40d5-9ba0-bcf2d8458644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:38.439678 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6eadb1-63cf-4dc0-9bc4-e2b368f51680 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:38.474217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 506/986] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:35:38.481655 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932ba11-7d80-488c-88e2-610f28832e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:38.482459 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932ba11-7d80-488c-88e2-610f28832e76 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:38.482750 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932ba11-7d80-488c-88e2-610f28832e76 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:38.483006 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932ba11-7d80-488c-88e2-610f28832e76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:38.494265 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932ba11-7d80-488c-88e2-610f28832e76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:38.495707 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932ba11-7d80-488c-88e2-610f28832e76 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:38.497300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686257b-bade-40d5-9ba0-bcf2d8458644 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:38.514131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-2932ba11-7d80-488c-88e2-610f28832e76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:38.514131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:38.514131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:38.514131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:38.514131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:38.514131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:38.514131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:38.514131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:38.514131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:38.514131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:38.514131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:38.514131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:38.514131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:38.514131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:38.514131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:38.515058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:38.515058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:38.515058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:38.515058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:38.515058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:38.515058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:38.515058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:38.515058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:38.515058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:38.515058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:38.515058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:38.515058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:38.515058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:38.515058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:38.515058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:38.515058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:38.515058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:38.520461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:38.520461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:38.520461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:38.520461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:38.520461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 481/987] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:38.522589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932ba11-7d80-488c-88e2-610f28832e76 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:38.567874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:38.599565 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686257b-bade-40d5-9ba0-bcf2d8458644 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:38.607994 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:38.637388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932ba11-7d80-488c-88e2-610f28832e76 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:38.637573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539f56f-53c4-41c9-9d7e-64c6c0e9cb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:38.638358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539f56f-53c4-41c9-9d7e-64c6c0e9cb41 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:38.638645 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539f56f-53c4-41c9-9d7e-64c6c0e9cb41 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:38.638924 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539f56f-53c4-41c9-9d7e-64c6c0e9cb41 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:38.639585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e539f56f-53c4-41c9-9d7e-64c6c0e9cb41 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:38.640937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e539f56f-53c4-41c9-9d7e-64c6c0e9cb41 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:38.653188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-e539f56f-53c4-41c9-9d7e-64c6c0e9cb41 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 15:35:38.653188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:38.653188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:38.653188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:38.653188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:38.653188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:38.653188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:38.653188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:38.653188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 15:35:38.653188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:38.653188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:38.653188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:38.653188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:38.653188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:38.653188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:38.653188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:38.653188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:38.654361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:38.654361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:38.654361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:38.654361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:38.654361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:38.654361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:38.654361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:38.654361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:38.654361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:38.654361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:38.654361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:38.654361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:38.654361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:38.654361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:38.654361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:38.654361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:38.655293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:38.655293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:38.655293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:38.655293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:38.655293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:38.655293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:38.655293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:38.655293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:38.655293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:38.655293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:38.655293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:38.655293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:38.655293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 15:35:38.655293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:38.655293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 507/988] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:38.660192 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539f56f-53c4-41c9-9d7e-64c6c0e9cb41 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:38.682555 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5abef-d2c0-4cb3-973c-09f0cd48dde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:38.683338 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5abef-d2c0-4cb3-973c-09f0cd48dde8 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:38.683686 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5abef-d2c0-4cb3-973c-09f0cd48dde8 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:38.684029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5abef-d2c0-4cb3-973c-09f0cd48dde8 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:38.684819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5abef-d2c0-4cb3-973c-09f0cd48dde8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:38.685193 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5abef-d2c0-4cb3-973c-09f0cd48dde8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:38.686668 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5abef-d2c0-4cb3-973c-09f0cd48dde8 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:38.699648 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 482/989] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:38.704621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5abef-d2c0-4cb3-973c-09f0cd48dde8 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:38.767685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539f56f-53c4-41c9-9d7e-64c6c0e9cb41 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:38.810989 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:38.814120 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8085bd-85ea-4d16-8e4b-88d28df7d451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:38.814795 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8085bd-85ea-4d16-8e4b-88d28df7d451 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:38.815079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8085bd-85ea-4d16-8e4b-88d28df7d451 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:38.815352 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8085bd-85ea-4d16-8e4b-88d28df7d451 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:38.815858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5abef-d2c0-4cb3-973c-09f0cd48dde8 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:38.816064 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8085bd-85ea-4d16-8e4b-88d28df7d451 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:38.816376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8085bd-85ea-4d16-8e4b-88d28df7d451 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:38.817772 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8085bd-85ea-4d16-8e4b-88d28df7d451 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:38.827617 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 508/990] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jun 27 15:35:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:38.830492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8085bd-85ea-4d16-8e4b-88d28df7d451 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:38.856394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ea77a-d6fd-449d-b345-a1c53fcbff47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:38.857200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ea77a-d6fd-449d-b345-a1c53fcbff47 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:38.857489 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ea77a-d6fd-449d-b345-a1c53fcbff47 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:38.857806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ea77a-d6fd-449d-b345-a1c53fcbff47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:38.864496 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ea77a-d6fd-449d-b345-a1c53fcbff47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:38.865988 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ea77a-d6fd-449d-b345-a1c53fcbff47 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:38.868624 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:38.882048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-610ea77a-d6fd-449d-b345-a1c53fcbff47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:38.882048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:38.882048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:38.882048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:38.882048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:38.882048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:38.882048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:38.882048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:38.882048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:38.882048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:38.882048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:38.882048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:38.882048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:38.882048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:38.882048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:38.882961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:38.882961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:38.882961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:38.882961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:38.882961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:38.882961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:38.882961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:38.882961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:38.882961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:38.882961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:38.882961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:38.882961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:38.882961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:38.882961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:38.882961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:38.882961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:38.882961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:38.884524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:38.884524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:38.884524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:38.884524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:38.884524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:38.884524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:38.884524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:38.884524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:38.884524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 483/991] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:38.890090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ea77a-d6fd-449d-b345-a1c53fcbff47 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:38.950449 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8085bd-85ea-4d16-8e4b-88d28df7d451 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:38.987214 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e183c-0f67-40d9-83d2-63de2f247f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:38.987981 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e183c-0f67-40d9-83d2-63de2f247f3e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:38.988375 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e183c-0f67-40d9-83d2-63de2f247f3e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:38.988653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e183c-0f67-40d9-83d2-63de2f247f3e None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:38.989355 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e183c-0f67-40d9-83d2-63de2f247f3e None admin] RBAC: Adding query filter params (type=compute) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:38.989692 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e183c-0f67-40d9-83d2-63de2f247f3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:38.991204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e183c-0f67-40d9-83d2-63de2f247f3e None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:38.994907 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ea77a-d6fd-449d-b345-a1c53fcbff47 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:38.997306 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 509/992] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jun 27 15:35:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:39.002956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e183c-0f67-40d9-83d2-63de2f247f3e None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:39.032320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195bddc-8324-4b3b-9567-0ef50e6df4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:39.033827 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195bddc-8324-4b3b-9567-0ef50e6df4c4 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:39.034186 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195bddc-8324-4b3b-9567-0ef50e6df4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:39.034464 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195bddc-8324-4b3b-9567-0ef50e6df4c4 None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:39.035312 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195bddc-8324-4b3b-9567-0ef50e6df4c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:39.035627 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195bddc-8324-4b3b-9567-0ef50e6df4c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:39.037360 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195bddc-8324-4b3b-9567-0ef50e6df4c4 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:39.051400 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 484/993] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:39.057728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195bddc-8324-4b3b-9567-0ef50e6df4c4 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:39.077283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:39.121595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:39.126284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e183c-0f67-40d9-83d2-63de2f247f3e None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:39.168027 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fbc75-0c2e-403f-a6ab-e6d8d13eaf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:39.168608 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fbc75-0c2e-403f-a6ab-e6d8d13eaf5c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:39.168896 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fbc75-0c2e-403f-a6ab-e6d8d13eaf5c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:39.169174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fbc75-0c2e-403f-a6ab-e6d8d13eaf5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:39.170007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fbc75-0c2e-403f-a6ab-e6d8d13eaf5c None admin] RBAC: Adding query filter params (interface=public, service_id=55f2e565ef394ba0bd23b8a3695bdbb0, region_id=RegionOne) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:39.170322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fbc75-0c2e-403f-a6ab-e6d8d13eaf5c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:39.171696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fbc75-0c2e-403f-a6ab-e6d8d13eaf5c None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:39.173914 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195bddc-8324-4b3b-9567-0ef50e6df4c4 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:39.177165 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 510/994] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Jun 27 15:35:39 2022] GET /identity/v3/endpoints?service_id=55f2e565ef394ba0bd23b8a3695bdbb0&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:39.184492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fbc75-0c2e-403f-a6ab-e6d8d13eaf5c None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:39.213212 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6caeeb-c4d9-429a-a6db-4a050ddc4abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:39.213905 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6caeeb-c4d9-429a-a6db-4a050ddc4abb None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:39.214192 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6caeeb-c4d9-429a-a6db-4a050ddc4abb None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:39.214472 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6caeeb-c4d9-429a-a6db-4a050ddc4abb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:39.215309 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6caeeb-c4d9-429a-a6db-4a050ddc4abb None admin] RBAC: Adding query filter params (role.id=507fb7d64e4549c58ff859abf6160c43, scope.project.id=7be83fb6f20341139147373837320148, user.id=6655366e7eb84a26aa3f7fbe88af62b5) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:39.215623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6caeeb-c4d9-429a-a6db-4a050ddc4abb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:39.217061 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6caeeb-c4d9-429a-a6db-4a050ddc4abb None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:39.229681 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 485/995] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:35:39 2022] GET /identity/v3/role_assignments?user.id=6655366e7eb84a26aa3f7fbe88af62b5&scope.project.id=7be83fb6f20341139147373837320148&role.id=507fb7d64e4549c58ff859abf6160c43 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:39.295977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fbc75-0c2e-403f-a6ab-e6d8d13eaf5c None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:39.332891 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4fd6de-7621-43ec-a30a-24ba6816d527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:39.333568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fd6de-7621-43ec-a30a-24ba6816d527 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:39.333881 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fd6de-7621-43ec-a30a-24ba6816d527 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:39.334161 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4fd6de-7621-43ec-a30a-24ba6816d527 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:39.334857 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fd6de-7621-43ec-a30a-24ba6816d527 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:39.336303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4fd6de-7621-43ec-a30a-24ba6816d527 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:39.344083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 511/996] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Jun 27 15:35:39 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:35:40.068133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2372e4-8c40-402d-a9d3-6de2f1abcce5 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:40.068643 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2372e4-8c40-402d-a9d3-6de2f1abcce5 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:40.068766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2372e4-8c40-402d-a9d3-6de2f1abcce5 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:40.070147 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 486/997] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:40.076098 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e170e-a4ac-454e-a8af-f2b5134101d3 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:40.076328 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e170e-a4ac-454e-a8af-f2b5134101d3 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:40.076607 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e170e-a4ac-454e-a8af-f2b5134101d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:40.094497 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-545e170e-a4ac-454e-a8af-f2b5134101d3 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:40.095671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-545e170e-a4ac-454e-a8af-f2b5134101d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:40.116750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:40.169656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 512/998] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:40.192682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2372e4-8c40-402d-a9d3-6de2f1abcce5 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:40.279223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:40.300320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2372e4-8c40-402d-a9d3-6de2f1abcce5 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:40.335704 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311007e9-8cb2-4e25-b7b6-9b18705ccefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:40.337784 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311007e9-8cb2-4e25-b7b6-9b18705ccefe None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:40.338082 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311007e9-8cb2-4e25-b7b6-9b18705ccefe None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:40.338362 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311007e9-8cb2-4e25-b7b6-9b18705ccefe None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:40.339021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311007e9-8cb2-4e25-b7b6-9b18705ccefe None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:40.340393 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311007e9-8cb2-4e25-b7b6-9b18705ccefe None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:40.353577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-311007e9-8cb2-4e25-b7b6-9b18705ccefe None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:35:40.353577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:40.353577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:40.353577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:40.353577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:40.353577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:40.353577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:40.353577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:40.353577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:35:40.353577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:40.353577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:40.353577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:40.353577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:40.353577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:40.353577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:40.353577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:40.353577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:40.354696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:40.354696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:40.354696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:40.354696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:40.354696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:40.354696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:40.354696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:40.354696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:40.354696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:40.354696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:40.354696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:40.354696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:40.354696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:40.354696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:40.354696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:40.354696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:40.355575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:40.355575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:40.355575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:40.355575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:40.355575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:40.355575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:40.355575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:40.355575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:40.355575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:40.355575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:40.355575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:40.355575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:40.355575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 15:35:40.355575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:40.357984 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 487/999] 10.222.0.20 () {56 vars in 1126 bytes} [Mon Jun 27 15:35:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:40.360672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e170e-a4ac-454e-a8af-f2b5134101d3 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:40.469651 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e170e-a4ac-454e-a8af-f2b5134101d3 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:40.506184 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2c7d3-e3a6-426a-9aad-a49171298b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:40.506993 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2c7d3-e3a6-426a-9aad-a49171298b8a None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:40.507123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2c7d3-e3a6-426a-9aad-a49171298b8a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:40.507404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2c7d3-e3a6-426a-9aad-a49171298b8a None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:40.508140 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2c7d3-e3a6-426a-9aad-a49171298b8a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:40.509067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2c7d3-e3a6-426a-9aad-a49171298b8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:40.510985 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2c7d3-e3a6-426a-9aad-a49171298b8a None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:40.519047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 513/1000] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jun 27 15:35:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:40.525017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311007e9-8cb2-4e25-b7b6-9b18705ccefe None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:40.628396 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311007e9-8cb2-4e25-b7b6-9b18705ccefe None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:40.660993 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb19d4-8339-4c95-92d5-c56aa668ba9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:40.661657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb19d4-8339-4c95-92d5-c56aa668ba9d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:40.661955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb19d4-8339-4c95-92d5-c56aa668ba9d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:40.662229 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb19d4-8339-4c95-92d5-c56aa668ba9d None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:40.662929 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb19d4-8339-4c95-92d5-c56aa668ba9d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:40.663234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb19d4-8339-4c95-92d5-c56aa668ba9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:40.665514 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb19d4-8339-4c95-92d5-c56aa668ba9d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:40.670820 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 488/1001] 10.222.0.20 () {56 vars in 1122 bytes} [Mon Jun 27 15:35:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:40.676302 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2c7d3-e3a6-426a-9aad-a49171298b8a None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:40.697816 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:40.780147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2c7d3-e3a6-426a-9aad-a49171298b8a None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:40.817139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ed24f-4321-42ca-861a-05de16842c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:40.817836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ed24f-4321-42ca-861a-05de16842c32 None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:40.818118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ed24f-4321-42ca-861a-05de16842c32 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:40.818390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ed24f-4321-42ca-861a-05de16842c32 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:40.819136 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ed24f-4321-42ca-861a-05de16842c32 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:40.821795 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ed24f-4321-42ca-861a-05de16842c32 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:40.849680 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 514/1002] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Jun 27 15:35:40 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:41.339993 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4f742-12a2-4180-b142-3dcdc9658963 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:41.340505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4f742-12a2-4180-b142-3dcdc9658963 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:41.340584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4f742-12a2-4180-b142-3dcdc9658963 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:41.342262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 489/1003] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:41.348007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ef1c9-bd70-42bd-b0d9-5334a12fa1d4 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:41.348435 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ef1c9-bd70-42bd-b0d9-5334a12fa1d4 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:41.348634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ef1c9-bd70-42bd-b0d9-5334a12fa1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:41.365936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-bd1ef1c9-bd70-42bd-b0d9-5334a12fa1d4 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:41.367131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-bd1ef1c9-bd70-42bd-b0d9-5334a12fa1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:41.412526 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd88ed7-a0e4-4a4b-a04d-231d298d801f None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:41.412811 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd88ed7-a0e4-4a4b-a04d-231d298d801f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:41.413094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd88ed7-a0e4-4a4b-a04d-231d298d801f None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:41.415140 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 490/1004] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:41.420011 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033bbfa6-ba29-4d66-adc7-b504261817a8 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:41.420430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033bbfa6-ba29-4d66-adc7-b504261817a8 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:41.420774 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033bbfa6-ba29-4d66-adc7-b504261817a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:41.437926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-033bbfa6-ba29-4d66-adc7-b504261817a8 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:41.439196 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-033bbfa6-ba29-4d66-adc7-b504261817a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:41.442106 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:41.447010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 515/1005] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:41 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jun 27 15:35:41.469057 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ef1c9-bd70-42bd-b0d9-5334a12fa1d4 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:41.477674 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:41.524868 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 491/1006] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:41 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:41.548744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033bbfa6-ba29-4d66-adc7-b504261817a8 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:41.581187 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ef1c9-bd70-42bd-b0d9-5334a12fa1d4 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:41.617998 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6a450-f24b-4d3c-b2dc-c1f7803d9598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:41.619077 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6a450-f24b-4d3c-b2dc-c1f7803d9598 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:41.619545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6a450-f24b-4d3c-b2dc-c1f7803d9598 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:41.620005 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6a450-f24b-4d3c-b2dc-c1f7803d9598 None admin] PATH_INFO: `/v3/services/image` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:41.620845 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6a450-f24b-4d3c-b2dc-c1f7803d9598 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:41.622245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6a450-f24b-4d3c-b2dc-c1f7803d9598 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:41.634729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-39f6a450-f24b-4d3c-b2dc-c1f7803d9598 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 15:35:41.634729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:41.634729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:41.634729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:41.634729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:41.634729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:41.634729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:41.634729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:41.634729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 15:35:41.634729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:41.634729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:41.634729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:41.634729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:41.634729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:41.634729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:41.634729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:41.634729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:41.636221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:41.636221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:41.636221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:41.636221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:41.636221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:41.636221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:41.636221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:41.636221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:41.636221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:41.636221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:41.636221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:41.636221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:41.636221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:41.636221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:41.636221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:41.636221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:41.637055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:41.637055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:41.637055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:41.637055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:41.637055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:41.637055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:41.637055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:41.637055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:41.637055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:41.637055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:41.637055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:41.637055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:41.637055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 15:35:41.637055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:41.637055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 516/1007] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Jun 27 15:35:41 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:41.641952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6a450-f24b-4d3c-b2dc-c1f7803d9598 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:41.655379 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033bbfa6-ba29-4d66-adc7-b504261817a8 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:41.691604 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c7d10-57e2-4069-b89a-be8e29c846df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:41.692576 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c7d10-57e2-4069-b89a-be8e29c846df None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:41.692942 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c7d10-57e2-4069-b89a-be8e29c846df None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:41.693299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c7d10-57e2-4069-b89a-be8e29c846df None admin] PATH_INFO: `/v3/services/compute` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:41.694154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4c7d10-57e2-4069-b89a-be8e29c846df None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:41.696438 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4c7d10-57e2-4069-b89a-be8e29c846df None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:41.711391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-5c4c7d10-57e2-4069-b89a-be8e29c846df None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 15:35:41.711391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:41.711391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:41.711391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:41.711391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:41.711391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:41.711391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:41.711391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:41.711391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 15:35:41.711391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:41.711391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:41.711391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:41.711391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:41.711391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:41.711391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:41.711391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:41.711391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:41.712397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:41.712397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:41.712397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:41.712397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:41.712397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:41.712397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:41.712397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:41.712397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:41.712397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:41.712397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:41.712397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:41.712397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:41.712397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:41.712397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:41.712397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:41.712397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:41.713258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:41.713258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:41.713258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:41.713258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:41.713258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:41.713258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:41.713258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:41.713258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:41.713258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:41.713258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:41.713258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:41.713258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:41.713258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 15:35:41.713258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:41.713258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 492/1008] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:41.716009 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:41.719275 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c7d10-57e2-4069-b89a-be8e29c846df None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:41.723782 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:41.752191 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6a450-f24b-4d3c-b2dc-c1f7803d9598 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:41.791195 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831da8cc-bc9b-4631-b610-9f868d892fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:41.792028 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831da8cc-bc9b-4631-b610-9f868d892fa7 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:41.792319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831da8cc-bc9b-4631-b610-9f868d892fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:41.793342 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831da8cc-bc9b-4631-b610-9f868d892fa7 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:41.793462 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831da8cc-bc9b-4631-b610-9f868d892fa7 None admin] RBAC: Adding query filter params (name=image) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:41.793738 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831da8cc-bc9b-4631-b610-9f868d892fa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:41.795124 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831da8cc-bc9b-4631-b610-9f868d892fa7 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:41.808418 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 517/1009] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:41.809115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831da8cc-bc9b-4631-b610-9f868d892fa7 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:41.829917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c7d10-57e2-4069-b89a-be8e29c846df None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:41.867309 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5c7e9-992b-4619-8228-85a8f72e3015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:41.868199 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5c7e9-992b-4619-8228-85a8f72e3015 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:41.868444 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5c7e9-992b-4619-8228-85a8f72e3015 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:41.868707 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5c7e9-992b-4619-8228-85a8f72e3015 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:41.869429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5c7e9-992b-4619-8228-85a8f72e3015 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:41.869773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5c7e9-992b-4619-8228-85a8f72e3015 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:41.871151 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5c7e9-992b-4619-8228-85a8f72e3015 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:41.880551 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 493/1010] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jun 27 15:35:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:41.881577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5c7e9-992b-4619-8228-85a8f72e3015 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:41.925499 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831da8cc-bc9b-4631-b610-9f868d892fa7 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:41.959271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:41.963686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0e76b-cf88-4e99-b3c3-5f6e964d0105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:41.964355 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0e76b-cf88-4e99-b3c3-5f6e964d0105 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:41.964634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0e76b-cf88-4e99-b3c3-5f6e964d0105 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:41.964907 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0e76b-cf88-4e99-b3c3-5f6e964d0105 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:41.965612 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0e76b-cf88-4e99-b3c3-5f6e964d0105 None admin] RBAC: Adding query filter params (type=image) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:41.965954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0e76b-cf88-4e99-b3c3-5f6e964d0105 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:41.967318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0e76b-cf88-4e99-b3c3-5f6e964d0105 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:41.968830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:41.973685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 518/1011] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:41 2022] GET /identity/v3/services?type=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:42.005680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5c7e9-992b-4619-8228-85a8f72e3015 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:42.044127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e165bf-6d49-472a-bfe6-6529bcc4692d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:42.044815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e165bf-6d49-472a-bfe6-6529bcc4692d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:42.045094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e165bf-6d49-472a-bfe6-6529bcc4692d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:42.045366 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e165bf-6d49-472a-bfe6-6529bcc4692d None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:42.046134 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e165bf-6d49-472a-bfe6-6529bcc4692d None admin] RBAC: Adding query filter params (type=compute) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:42.046443 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e165bf-6d49-472a-bfe6-6529bcc4692d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:42.047838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e165bf-6d49-472a-bfe6-6529bcc4692d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:42.053746 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 494/1012] 10.222.0.20 () {56 vars in 1112 bytes} [Mon Jun 27 15:35:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:42.060124 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0e76b-cf88-4e99-b3c3-5f6e964d0105 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:42.182026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0e76b-cf88-4e99-b3c3-5f6e964d0105 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:42.220266 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31508b3-44fe-4fda-a492-0e7b1ea2092d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:42.220923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31508b3-44fe-4fda-a492-0e7b1ea2092d None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:42.221207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31508b3-44fe-4fda-a492-0e7b1ea2092d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:42.221473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31508b3-44fe-4fda-a492-0e7b1ea2092d None admin] PATH_INFO: `/v3/endpoints` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:42.222261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31508b3-44fe-4fda-a492-0e7b1ea2092d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:42.223616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31508b3-44fe-4fda-a492-0e7b1ea2092d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:42.249674 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 519/1013] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Jun 27 15:35:42 2022] POST /identity/v3/endpoints => generated 334 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:42.956621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823f469-94f4-4a97-99c6-0d895baea286 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:42.957130 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823f469-94f4-4a97-99c6-0d895baea286 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:42.957229 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823f469-94f4-4a97-99c6-0d895baea286 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:42.958716 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 495/1014] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:42.964910 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3318b-31b7-4b5a-aaf5-72f2b5f1d00f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:42.965300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3318b-31b7-4b5a-aaf5-72f2b5f1d00f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:42.965459 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3318b-31b7-4b5a-aaf5-72f2b5f1d00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:42.983107 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-1dc3318b-31b7-4b5a-aaf5-72f2b5f1d00f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:42.984301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1dc3318b-31b7-4b5a-aaf5-72f2b5f1d00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:43.035180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:43.065786 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 520/1015] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:42 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:43.087835 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823f469-94f4-4a97-99c6-0d895baea286 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:43.198774 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823f469-94f4-4a97-99c6-0d895baea286 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:43.228458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:43.234435 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a94cd6d-08d9-45db-aded-45f2141f45ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:43.235226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94cd6d-08d9-45db-aded-45f2141f45ad None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:43.235508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94cd6d-08d9-45db-aded-45f2141f45ad None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:43.235782 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94cd6d-08d9-45db-aded-45f2141f45ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:43.236953 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94cd6d-08d9-45db-aded-45f2141f45ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:43.238427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a94cd6d-08d9-45db-aded-45f2141f45ad None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:43.240255 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 496/1016] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:43.246470 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3318b-31b7-4b5a-aaf5-72f2b5f1d00f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:43.349691 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc3318b-31b7-4b5a-aaf5-72f2b5f1d00f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:43.385261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c729ad1-9d7b-438d-824e-dff63f532cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:43.385970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c729ad1-9d7b-438d-824e-dff63f532cb9 None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:43.386300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c729ad1-9d7b-438d-824e-dff63f532cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:43.386581 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c729ad1-9d7b-438d-824e-dff63f532cb9 None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:43.387297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c729ad1-9d7b-438d-824e-dff63f532cb9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:43.388824 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c729ad1-9d7b-438d-824e-dff63f532cb9 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:43.421674 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 521/1017] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jun 27 15:35:43 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:44.046336 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d96f80-53f8-4e56-97f8-8b946e2ad9ef None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:44.046850 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d96f80-53f8-4e56-97f8-8b946e2ad9ef None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:44.046926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d96f80-53f8-4e56-97f8-8b946e2ad9ef None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:44.048487 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 497/1018] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:44.053459 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21548bd9-d878-4f7e-b41f-c2f054d0d6b3 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:44.054112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21548bd9-d878-4f7e-b41f-c2f054d0d6b3 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:44.054112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21548bd9-d878-4f7e-b41f-c2f054d0d6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:44.071508 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-21548bd9-d878-4f7e-b41f-c2f054d0d6b3 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:44.072804 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-21548bd9-d878-4f7e-b41f-c2f054d0d6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:44.101822 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:44.146605 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 522/1019] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:44 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:44.170328 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d96f80-53f8-4e56-97f8-8b946e2ad9ef None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:44.281000 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d96f80-53f8-4e56-97f8-8b946e2ad9ef None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:44.316060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470b5c2-3bdb-4ac8-9bb3-0a44d8c59c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:44.316733 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470b5c2-3bdb-4ac8-9bb3-0a44d8c59c87 None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:44.317020 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470b5c2-3bdb-4ac8-9bb3-0a44d8c59c87 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:44.317299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470b5c2-3bdb-4ac8-9bb3-0a44d8c59c87 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:44.318067 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470b5c2-3bdb-4ac8-9bb3-0a44d8c59c87 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:44.319434 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470b5c2-3bdb-4ac8-9bb3-0a44d8c59c87 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:44.341685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 498/1020] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jun 27 15:35:44 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:35:44.472779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2047ea-4d3e-4db1-976e-3780e423b40f None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:44.473097 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2047ea-4d3e-4db1-976e-3780e423b40f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:44.473407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2047ea-4d3e-4db1-976e-3780e423b40f None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:44.475377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 523/1021] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:44.481856 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d11a74-e471-4ede-895c-f7d542d1464b None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:44.482146 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d11a74-e471-4ede-895c-f7d542d1464b None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:44.482433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d11a74-e471-4ede-895c-f7d542d1464b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:44.499855 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-27d11a74-e471-4ede-895c-f7d542d1464b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:44.501055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-27d11a74-e471-4ede-895c-f7d542d1464b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:44.569381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:44.591239 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 499/1022] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:44.615632 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2047ea-4d3e-4db1-976e-3780e423b40f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:44.724701 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2047ea-4d3e-4db1-976e-3780e423b40f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:44.760309 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154e329-ab5e-4c38-8bef-ec04a5b9191e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:44.761118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154e329-ab5e-4c38-8bef-ec04a5b9191e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:44.761403 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154e329-ab5e-4c38-8bef-ec04a5b9191e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:44.761725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154e329-ab5e-4c38-8bef-ec04a5b9191e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:44.771961 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154e329-ab5e-4c38-8bef-ec04a5b9191e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:44.780287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-1154e329-ab5e-4c38-8bef-ec04a5b9191e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:35:44.780287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:44.780287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:44.780287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:44.780287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:44.780287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:44.780287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:44.780287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:44.780287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:44.780287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:44.780287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:44.780287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:44.780287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:44.780287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:44.780287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:44.780287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:44.781358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:44.781358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:44.781358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:44.781358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:44.781358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:44.781358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:44.781358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:44.781358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:44.781358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:44.781358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:44.781358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:44.781358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:44.781358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:44.781358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:44.781358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:44.781358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:44.782228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:44.782228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:44.782228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:44.782228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:44.782228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:44.782228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:44.782228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:44.782228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:44.782228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:44.782228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:44.782228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:35:44.782228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:44.785706 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 524/1023] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Jun 27 15:35:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:44.787695 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d11a74-e471-4ede-895c-f7d542d1464b None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:44.915691 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d11a74-e471-4ede-895c-f7d542d1464b None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:44.952271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192aef2c-a641-47d1-992e-7a7ca438c214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:44.952956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192aef2c-a641-47d1-992e-7a7ca438c214 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:44.953242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192aef2c-a641-47d1-992e-7a7ca438c214 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:44.953521 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192aef2c-a641-47d1-992e-7a7ca438c214 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:44.954302 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192aef2c-a641-47d1-992e-7a7ca438c214 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:44.954614 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192aef2c-a641-47d1-992e-7a7ca438c214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:44.955989 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192aef2c-a641-47d1-992e-7a7ca438c214 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:44.969754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 500/1024] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Jun 27 15:35:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:44.971447 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154e329-ab5e-4c38-8bef-ec04a5b9191e None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:44.998397 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:45.081343 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154e329-ab5e-4c38-8bef-ec04a5b9191e None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:45.115823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6bc73-0b1c-4dcd-bff5-acd25e95bc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:45.116662 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6bc73-0b1c-4dcd-bff5-acd25e95bc58 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:45.116951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6bc73-0b1c-4dcd-bff5-acd25e95bc58 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:45.117227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6bc73-0b1c-4dcd-bff5-acd25e95bc58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:45.129098 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6bc73-0b1c-4dcd-bff5-acd25e95bc58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:45.130574 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f6bc73-0b1c-4dcd-bff5-acd25e95bc58 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:45.148892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-d7f6bc73-0b1c-4dcd-bff5-acd25e95bc58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:45.148892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:45.148892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:45.148892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:45.148892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:45.148892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:45.148892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:45.148892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:45.148892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:45.148892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:45.148892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:45.148892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:45.148892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:45.148892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:45.148892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:45.150286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:45.150286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:45.150286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:45.150286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:45.150286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:45.150286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:45.150286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:45.150286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:45.150286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:45.150286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:45.150286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:45.150286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:45.150286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:45.150286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:45.150286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:45.150286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:45.150286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:45.151188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:45.151188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:45.151188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:45.151188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:45.154791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 525/1025] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:45.156150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192aef2c-a641-47d1-992e-7a7ca438c214 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:45.213248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:45.272693 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192aef2c-a641-47d1-992e-7a7ca438c214 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:45.311098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b229e-6614-46a4-8393-d5c023042234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:45.311784 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b229e-6614-46a4-8393-d5c023042234 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:45.312071 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b229e-6614-46a4-8393-d5c023042234 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:45.312348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b229e-6614-46a4-8393-d5c023042234 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:45.313095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b229e-6614-46a4-8393-d5c023042234 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:45.313595 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b229e-6614-46a4-8393-d5c023042234 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:45.315057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b229e-6614-46a4-8393-d5c023042234 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:45.332326 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 501/1026] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:45.333931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6bc73-0b1c-4dcd-bff5-acd25e95bc58 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:45.412352 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:45.439788 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6bc73-0b1c-4dcd-bff5-acd25e95bc58 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:45.482694 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8dc332-ed9b-4218-977c-2deb46280f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:45.483488 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8dc332-ed9b-4218-977c-2deb46280f25 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:45.483773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8dc332-ed9b-4218-977c-2deb46280f25 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:45.484049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8dc332-ed9b-4218-977c-2deb46280f25 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:45.492053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8dc332-ed9b-4218-977c-2deb46280f25 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:45.493613 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8dc332-ed9b-4218-977c-2deb46280f25 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:45.509479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-7d8dc332-ed9b-4218-977c-2deb46280f25 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:35:45.509479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:45.509479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:45.509479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:45.509479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:45.509479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:45.509479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:45.509479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:45.509479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:45.509479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:45.509479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:45.509479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:45.509479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:35:45.509479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:35:45.509479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:35:45.509479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:35:45.510639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:45.510639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:45.510639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:45.510639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:45.510639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:45.510639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:45.510639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:45.510639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:45.510639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:45.510639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:45.510639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:45.510639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:45.510639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:45.510639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:45.510639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:45.510639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:45.510639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:45.510639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:45.516376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:45.516376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:45.516376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:35:45.516376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:35:45.516376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:35:45.516376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:35:45.516376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:35:45.516376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:35:45.516376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:35:45.516376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:45.516376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 526/1027] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Jun 27 15:35:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:45.518429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b229e-6614-46a4-8393-d5c023042234 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:45.569333 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40960a-7bc4-4510-a15a-aca0109aaf76 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:45.569630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40960a-7bc4-4510-a15a-aca0109aaf76 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:45.569847 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40960a-7bc4-4510-a15a-aca0109aaf76 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:45.572277 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 527/1028] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:45.578171 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc459975-6546-466c-85f0-0d1b4b1a1542 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:45.578456 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc459975-6546-466c-85f0-0d1b4b1a1542 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:45.578778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc459975-6546-466c-85f0-0d1b4b1a1542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:45.597987 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-bc459975-6546-466c-85f0-0d1b4b1a1542 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:45.599163 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-bc459975-6546-466c-85f0-0d1b4b1a1542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:45.639873 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b229e-6614-46a4-8393-d5c023042234 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:45.652820 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:45.677187 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 528/1029] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:45.684947 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e3047-f516-494c-b61b-2ee10ecf3d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:45.685625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e3047-f516-494c-b61b-2ee10ecf3d60 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:45.685938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e3047-f516-494c-b61b-2ee10ecf3d60 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:45.686216 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e3047-f516-494c-b61b-2ee10ecf3d60 None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:45.687053 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e3047-f516-494c-b61b-2ee10ecf3d60 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:45.687363 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e3047-f516-494c-b61b-2ee10ecf3d60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:45.688811 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e3047-f516-494c-b61b-2ee10ecf3d60 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:45.702282 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc459975-6546-466c-85f0-0d1b4b1a1542 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:45.712860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 502/1030] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Jun 27 15:35:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:45.713963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e3047-f516-494c-b61b-2ee10ecf3d60 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:45.769529 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:45.814913 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc459975-6546-466c-85f0-0d1b4b1a1542 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:45.826513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e3047-f516-494c-b61b-2ee10ecf3d60 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:45.854037 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28bedc-e9c3-4ddb-90e4-9941d4360e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:45.855531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28bedc-e9c3-4ddb-90e4-9941d4360e5b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:45.855942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28bedc-e9c3-4ddb-90e4-9941d4360e5b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:45.856275 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28bedc-e9c3-4ddb-90e4-9941d4360e5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:45.858172 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e28bedc-e9c3-4ddb-90e4-9941d4360e5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:45.860400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e28bedc-e9c3-4ddb-90e4-9941d4360e5b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:45.863002 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 529/1031] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:45.871949 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28bedc-e9c3-4ddb-90e4-9941d4360e5b None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:45.876470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc28092-ed59-49be-a15a-e5ef4f75a4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:45.877215 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc28092-ed59-49be-a15a-e5ef4f75a4f9 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:45.877495 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc28092-ed59-49be-a15a-e5ef4f75a4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:45.878275 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc28092-ed59-49be-a15a-e5ef4f75a4f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:45.922620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:45.964374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc28092-ed59-49be-a15a-e5ef4f75a4f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:45.965875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc28092-ed59-49be-a15a-e5ef4f75a4f9 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:45.986175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28bedc-e9c3-4ddb-90e4-9941d4360e5b None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:46.018815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:46.026304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b01753-7ee3-40d8-a691-73197a3787fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:46.027082 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b01753-7ee3-40d8-a691-73197a3787fc None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.027364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b01753-7ee3-40d8-a691-73197a3787fc None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.027681 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b01753-7ee3-40d8-a691-73197a3787fc None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.028461 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b01753-7ee3-40d8-a691-73197a3787fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:46.030072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b01753-7ee3-40d8-a691-73197a3787fc None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:46.052588 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 530/1032] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jun 27 15:35:45 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:35:46.081121 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 503/1033] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:35:45 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 15:35:46.087205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b01753-7ee3-40d8-a691-73197a3787fc None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:46.199064 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b01753-7ee3-40d8-a691-73197a3787fc None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:46.236644 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fe238-e3c2-40dd-8354-b35cbce7d3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:46.237445 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fe238-e3c2-40dd-8354-b35cbce7d3d1 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.237784 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fe238-e3c2-40dd-8354-b35cbce7d3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.238035 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fe238-e3c2-40dd-8354-b35cbce7d3d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.252332 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46fe238-e3c2-40dd-8354-b35cbce7d3d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:46.254560 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46fe238-e3c2-40dd-8354-b35cbce7d3d1 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:46.272873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-f46fe238-e3c2-40dd-8354-b35cbce7d3d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:46.272873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:46.272873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:46.272873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:46.272873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:46.272873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:46.272873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:46.272873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:46.272873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:46.272873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:46.272873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:46.272873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:46.272873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:46.272873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:46.272873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:46.274892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:46.274892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:46.274892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:46.274892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:46.274892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:46.274892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:46.274892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:46.274892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:46.274892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:46.274892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:46.274892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:46.274892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:46.274892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:46.274892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:46.274892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:46.274892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:46.274892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:46.275769 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:46.275769 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:46.275769 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:46.275769 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:46.275769 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 531/1034] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:46.280317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc28092-ed59-49be-a15a-e5ef4f75a4f9 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:46.354355 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:46.394794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc28092-ed59-49be-a15a-e5ef4f75a4f9 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:46.434815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273e7fd-52bc-4bc3-958e-2f8bf4e6ac32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:46.435486 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273e7fd-52bc-4bc3-958e-2f8bf4e6ac32 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.435771 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273e7fd-52bc-4bc3-958e-2f8bf4e6ac32 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.436048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273e7fd-52bc-4bc3-958e-2f8bf4e6ac32 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.436784 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273e7fd-52bc-4bc3-958e-2f8bf4e6ac32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:46.437095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273e7fd-52bc-4bc3-958e-2f8bf4e6ac32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:46.438517 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273e7fd-52bc-4bc3-958e-2f8bf4e6ac32 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:46.457668 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 504/1035] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:46.457951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab748806-6e5e-4471-8f58-26ae14ea3b83 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.457951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab748806-6e5e-4471-8f58-26ae14ea3b83 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.457951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab748806-6e5e-4471-8f58-26ae14ea3b83 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.458188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 532/1036] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:46.460948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273e7fd-52bc-4bc3-958e-2f8bf4e6ac32 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:46.464229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dccb96-1490-493d-8c3b-6398c610fef0 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.464518 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dccb96-1490-493d-8c3b-6398c610fef0 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.464799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dccb96-1490-493d-8c3b-6398c610fef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.482241 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-44dccb96-1490-493d-8c3b-6398c610fef0 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:46.483450 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-44dccb96-1490-493d-8c3b-6398c610fef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:46.520516 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:46.558098 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 533/1037] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:35:46.577066 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273e7fd-52bc-4bc3-958e-2f8bf4e6ac32 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:46.581072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dccb96-1490-493d-8c3b-6398c610fef0 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:46.607426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:46.620554 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85565057-62d3-4b19-ab76-0f93518b49a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:46.621362 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85565057-62d3-4b19-ab76-0f93518b49a4 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.621682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85565057-62d3-4b19-ab76-0f93518b49a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.622417 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85565057-62d3-4b19-ab76-0f93518b49a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.630403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85565057-62d3-4b19-ab76-0f93518b49a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:46.631862 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85565057-62d3-4b19-ab76-0f93518b49a4 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:46.647300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-85565057-62d3-4b19-ab76-0f93518b49a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:46.647300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:46.647300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:46.647300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:46.647300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:46.647300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:46.647300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:46.647300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:46.647300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:46.647300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:46.647300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:46.647300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:46.647300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:46.647300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:46.647300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:46.648558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:46.648558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:46.648558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:46.648558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:46.648558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:46.648558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:46.648558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:46.648558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:46.648558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:46.648558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:46.648558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:46.648558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:46.648558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:46.648558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:46.648558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:46.648558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:46.648558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:46.650545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:46.650545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:46.650545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:46.650545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:46.650545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:46.650545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:46.650545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:46.650545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:46.650545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 505/1038] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:46.654750 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85565057-62d3-4b19-ab76-0f93518b49a4 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:46.695020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dccb96-1490-493d-8c3b-6398c610fef0 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:46.733877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950914b0-a661-4617-bb86-d1766f54a50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:46.734702 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950914b0-a661-4617-bb86-d1766f54a50b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.735062 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950914b0-a661-4617-bb86-d1766f54a50b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.735339 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950914b0-a661-4617-bb86-d1766f54a50b None admin] PATH_INFO: `/v3/services/image` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.735991 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950914b0-a661-4617-bb86-d1766f54a50b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:46.737472 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950914b0-a661-4617-bb86-d1766f54a50b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:46.750969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-950914b0-a661-4617-bb86-d1766f54a50b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 15:35:46.750969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:46.750969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:46.750969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:46.750969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:46.750969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:46.750969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:46.750969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:46.750969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 15:35:46.750969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:46.750969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:46.750969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:46.750969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:46.750969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:46.750969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:46.750969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:46.750969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:46.752059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:46.752059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:46.752059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:46.752059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:46.752059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:46.752059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:46.752059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:46.752059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:46.752059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:46.752059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:46.752059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:46.752059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:46.752059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:46.752059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:46.752059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:46.752059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:46.752920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:46.752920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:46.752920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:46.752920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:46.752920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:46.752920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:46.752920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:46.752920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:46.752920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:46.752920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:46.752920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:46.752920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:46.752920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 15:35:46.752920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:46.752920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 534/1039] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Jun 27 15:35:46 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:46.758732 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950914b0-a661-4617-bb86-d1766f54a50b None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:46.761539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85565057-62d3-4b19-ab76-0f93518b49a4 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:46.796258 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:46.796868 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79d7b6-129d-4ee0-aeba-05d92b774cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:46.797558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79d7b6-129d-4ee0-aeba-05d92b774cd8 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.797833 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79d7b6-129d-4ee0-aeba-05d92b774cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.798408 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79d7b6-129d-4ee0-aeba-05d92b774cd8 None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.799025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79d7b6-129d-4ee0-aeba-05d92b774cd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:46.799347 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79d7b6-129d-4ee0-aeba-05d92b774cd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:46.800811 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79d7b6-129d-4ee0-aeba-05d92b774cd8 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:46.819702 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 506/1040] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:46.822100 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79d7b6-129d-4ee0-aeba-05d92b774cd8 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:46.877970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:46.878159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950914b0-a661-4617-bb86-d1766f54a50b None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:46.920385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f516c3-b747-4f37-96b4-13ca9a9c0e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:46.921293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f516c3-b747-4f37-96b4-13ca9a9c0e90 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.921755 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f516c3-b747-4f37-96b4-13ca9a9c0e90 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.922177 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f516c3-b747-4f37-96b4-13ca9a9c0e90 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.923030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f516c3-b747-4f37-96b4-13ca9a9c0e90 None admin] RBAC: Adding query filter params (name=image) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:46.923511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f516c3-b747-4f37-96b4-13ca9a9c0e90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:46.925020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f516c3-b747-4f37-96b4-13ca9a9c0e90 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:46.930582 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 535/1041] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:46.936088 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f516c3-b747-4f37-96b4-13ca9a9c0e90 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:46.947806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79d7b6-129d-4ee0-aeba-05d92b774cd8 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:46.982252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a3ffe-6820-491d-9ca4-80bdecb627bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:46.982913 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a3ffe-6820-491d-9ca4-80bdecb627bf None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:46.983197 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a3ffe-6820-491d-9ca4-80bdecb627bf None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:46.983473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a3ffe-6820-491d-9ca4-80bdecb627bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:46.984340 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a3ffe-6820-491d-9ca4-80bdecb627bf None admin] RBAC: Adding query filter params (role.id=507fb7d64e4549c58ff859abf6160c43, scope.project.id=7be83fb6f20341139147373837320148, user.id=1c8f6262cf24419daf803ae71aed24f5) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:46.984654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a3ffe-6820-491d-9ca4-80bdecb627bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:46.986137 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a3ffe-6820-491d-9ca4-80bdecb627bf None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:46.991668 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 507/1042] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:35:46 2022] GET /identity/v3/role_assignments?user.id=1c8f6262cf24419daf803ae71aed24f5&scope.project.id=7be83fb6f20341139147373837320148&role.id=507fb7d64e4549c58ff859abf6160c43 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:47.058064 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:47.064076 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f516c3-b747-4f37-96b4-13ca9a9c0e90 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:47.105089 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ee9fc-03b7-4964-99d7-d76f2eabf1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:47.105908 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ee9fc-03b7-4964-99d7-d76f2eabf1fb None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:47.106096 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ee9fc-03b7-4964-99d7-d76f2eabf1fb None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:47.106376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ee9fc-03b7-4964-99d7-d76f2eabf1fb None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:47.107097 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ee9fc-03b7-4964-99d7-d76f2eabf1fb None admin] RBAC: Adding query filter params (type=image) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:47.107407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ee9fc-03b7-4964-99d7-d76f2eabf1fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:47.108821 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ee9fc-03b7-4964-99d7-d76f2eabf1fb None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:47.118304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 536/1043] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:47.121533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a3ffe-6820-491d-9ca4-80bdecb627bf None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:47.237713 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a3ffe-6820-491d-9ca4-80bdecb627bf None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:47.250884 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:47.279412 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35acb7-4f47-4faf-a94f-91dd7412a028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:47.280253 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35acb7-4f47-4faf-a94f-91dd7412a028 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:47.280656 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35acb7-4f47-4faf-a94f-91dd7412a028 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:47.281516 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35acb7-4f47-4faf-a94f-91dd7412a028 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:47.282345 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35acb7-4f47-4faf-a94f-91dd7412a028 None admin] RBAC: Adding query filter params (interface=public, service_id=7aafeb59d8bd443280ef02e216c08bef, region_id=RegionOne) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:47.282791 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35acb7-4f47-4faf-a94f-91dd7412a028 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:47.284395 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35acb7-4f47-4faf-a94f-91dd7412a028 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:47.292353 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 508/1044] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Jun 27 15:35:47 2022] GET /identity/v3/endpoints?service_id=7aafeb59d8bd443280ef02e216c08bef&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:47.298141 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ee9fc-03b7-4964-99d7-d76f2eabf1fb None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:47.471922 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ee9fc-03b7-4964-99d7-d76f2eabf1fb None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:47.529513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e86050-2e64-46f5-84c2-9d823bdef022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:47.530938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e86050-2e64-46f5-84c2-9d823bdef022 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:47.531628 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e86050-2e64-46f5-84c2-9d823bdef022 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:47.532322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e86050-2e64-46f5-84c2-9d823bdef022 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:47.533861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e86050-2e64-46f5-84c2-9d823bdef022 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:47.536612 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e86050-2e64-46f5-84c2-9d823bdef022 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:47.550574 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 537/1045] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Jun 27 15:35:47 2022] GET /identity/v3/services => generated 1957 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:35:48.693958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be2586-1078-40dd-9834-8fee32dc84de None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:48.694557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be2586-1078-40dd-9834-8fee32dc84de None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:48.694941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be2586-1078-40dd-9834-8fee32dc84de None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:48.698317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 509/1046] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:48.705404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e240ea-4051-4379-be98-d6f5b5c420fe None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:48.705865 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e240ea-4051-4379-be98-d6f5b5c420fe None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:48.706031 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e240ea-4051-4379-be98-d6f5b5c420fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:48.723460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-52e240ea-4051-4379-be98-d6f5b5c420fe None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:48.724658 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-52e240ea-4051-4379-be98-d6f5b5c420fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:48.745699 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:48.800216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 538/1047] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:48.819623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be2586-1078-40dd-9834-8fee32dc84de None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:48.932890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be2586-1078-40dd-9834-8fee32dc84de None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:48.967972 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f00be5-7c43-4c03-8942-b71184f6c4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:48.968995 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f00be5-7c43-4c03-8942-b71184f6c4ac None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:48.969429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f00be5-7c43-4c03-8942-b71184f6c4ac None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:48.969894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f00be5-7c43-4c03-8942-b71184f6c4ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:48.981784 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f00be5-7c43-4c03-8942-b71184f6c4ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:48.989584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-a3f00be5-7c43-4c03-8942-b71184f6c4ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:35:48.989584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:48.989584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:48.989584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:48.989584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:48.989584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:48.989584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:48.989584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:48.989584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:48.989584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:48.989584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:48.989584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:48.989584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:48.989584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:48.989584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:48.989584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:48.990622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:48.990622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:48.990622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:48.990622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:48.990622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:48.990622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:48.990622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:48.990622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:48.990622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:48.990622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:48.990622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:48.990622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:48.990622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:48.990622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:48.990622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:48.990622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:48.991458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:48.991458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:48.991458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:48.991458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:48.991458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:48.991458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:48.991458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:48.991458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:48.991458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:48.991458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:48.991458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:35:48.991458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:48.992101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 510/1048] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:35:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:48.998354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e240ea-4051-4379-be98-d6f5b5c420fe None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:49.100420 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e240ea-4051-4379-be98-d6f5b5c420fe None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:49.137208 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690594e-47db-44ff-8a07-efc53234bf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:49.137917 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690594e-47db-44ff-8a07-efc53234bf04 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:49.138204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690594e-47db-44ff-8a07-efc53234bf04 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:49.138481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690594e-47db-44ff-8a07-efc53234bf04 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:49.139233 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690594e-47db-44ff-8a07-efc53234bf04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:49.139544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690594e-47db-44ff-8a07-efc53234bf04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:49.140910 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690594e-47db-44ff-8a07-efc53234bf04 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:49.155007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 539/1049] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:35:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:49.156193 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f00be5-7c43-4c03-8942-b71184f6c4ac None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:49.191057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:49.267332 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f00be5-7c43-4c03-8942-b71184f6c4ac None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:49.312175 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb3ce7-3965-4cf1-b6d3-b9874aead239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:49.313061 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb3ce7-3965-4cf1-b6d3-b9874aead239 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:49.313346 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb3ce7-3965-4cf1-b6d3-b9874aead239 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:49.313626 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb3ce7-3965-4cf1-b6d3-b9874aead239 None admin] PATH_INFO: `/v3/users/bb6dd937abbd432e946d06713440c7d1` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:49.328916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb3ce7-3965-4cf1-b6d3-b9874aead239 None admin] RBAC: Authorizing `identity:get_user(user_id=bb6dd937abbd432e946d06713440c7d1)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:49.330397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb3ce7-3965-4cf1-b6d3-b9874aead239 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:49.337305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 511/1050] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:35:49 2022] GET /identity/v3/users/bb6dd937abbd432e946d06713440c7d1 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:49.338594 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690594e-47db-44ff-8a07-efc53234bf04 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:49.365358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:49.424929 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb8b86-cf05-48d2-90b3-a20510d576d2 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:49.425218 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb8b86-cf05-48d2-90b3-a20510d576d2 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:49.425498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb8b86-cf05-48d2-90b3-a20510d576d2 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:49.426921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 512/1051] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:49.433388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa58b7e-848e-4b96-b7bf-3b94cd6cdca7 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:49.433937 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa58b7e-848e-4b96-b7bf-3b94cd6cdca7 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:49.434468 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa58b7e-848e-4b96-b7bf-3b94cd6cdca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:49.454330 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-faa58b7e-848e-4b96-b7bf-3b94cd6cdca7 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:49.455625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-faa58b7e-848e-4b96-b7bf-3b94cd6cdca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:49.466481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690594e-47db-44ff-8a07-efc53234bf04 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:49.502600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180cf3e-4e59-4f73-9fda-18027911bad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:49.503383 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180cf3e-4e59-4f73-9fda-18027911bad9 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:49.503666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180cf3e-4e59-4f73-9fda-18027911bad9 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:49.503945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180cf3e-4e59-4f73-9fda-18027911bad9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:49.533796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 513/1052] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:35:49.552676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa58b7e-848e-4b96-b7bf-3b94cd6cdca7 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:49.579318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:49.579622 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180cf3e-4e59-4f73-9fda-18027911bad9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:49.581041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180cf3e-4e59-4f73-9fda-18027911bad9 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:49.600201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:49.660286 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa58b7e-848e-4b96-b7bf-3b94cd6cdca7 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:49.687561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 540/1053] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:35:49 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:35:49.693716 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f4e2a-1304-47d8-a6b3-fcbc8b6becb7 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:49.694012 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f4e2a-1304-47d8-a6b3-fcbc8b6becb7 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:49.694291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f4e2a-1304-47d8-a6b3-fcbc8b6becb7 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:49.694785 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82110e-107e-48b4-a0c3-5245b9585868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:49.696154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 541/1054] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:49.696295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82110e-107e-48b4-a0c3-5245b9585868 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:49.696427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82110e-107e-48b4-a0c3-5245b9585868 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:49.696699 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82110e-107e-48b4-a0c3-5245b9585868 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:49.696807 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f4e2a-1304-47d8-a6b3-fcbc8b6becb7 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:49.709034 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82110e-107e-48b4-a0c3-5245b9585868 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:49.718023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-3d82110e-107e-48b4-a0c3-5245b9585868 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:35:49.718023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:49.718023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:49.718023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:49.718023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:49.718023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:49.718023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:49.718023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:49.718023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:49.718023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:49.718023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:49.718023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:49.718023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:49.718023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:49.718023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:49.718023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:49.718965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:49.718965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:49.718965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:49.718965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:49.718965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:49.718965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:49.718965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:49.718965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:49.718965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:49.718965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:49.718965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:49.718965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:49.718965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:49.718965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:49.718965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:49.718965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:49.719970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:49.719970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:49.719970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:49.719970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:49.719970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:49.719970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:49.719970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:49.719970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:49.719970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:49.719970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:49.719970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:35:49.719970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:49.719970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 514/1055] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Jun 27 15:35:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:49.721566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2847f-e3ec-4647-aacb-f3e382419dda None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:49.721566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2847f-e3ec-4647-aacb-f3e382419dda None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:49.721566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2847f-e3ec-4647-aacb-f3e382419dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:49.737625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-5af2847f-e3ec-4647-aacb-f3e382419dda None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:49.738825 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-5af2847f-e3ec-4647-aacb-f3e382419dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:49.803322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f4e2a-1304-47d8-a6b3-fcbc8b6becb7 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:49.811734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 515/1056] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:49.813017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2847f-e3ec-4647-aacb-f3e382419dda None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:49.823076 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:49.841317 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172fa0c-4fab-4142-ae45-74f4436de844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:49.842213 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172fa0c-4fab-4142-ae45-74f4436de844 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:49.842498 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172fa0c-4fab-4142-ae45-74f4436de844 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:49.842774 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172fa0c-4fab-4142-ae45-74f4436de844 None admin] PATH_INFO: `/v3/projects/6fddfead5e6c40e289cd1973ec85c380` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:49.843957 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172fa0c-4fab-4142-ae45-74f4436de844 None admin] RBAC: Authorizing `identity:get_project(project_id=6fddfead5e6c40e289cd1973ec85c380)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:49.844761 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:49.845411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172fa0c-4fab-4142-ae45-74f4436de844 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:49.847314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 542/1057] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:35:49 2022] GET /identity/v3/projects/6fddfead5e6c40e289cd1973ec85c380 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:49.847920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172fa0c-4fab-4142-ae45-74f4436de844 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:49.934236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2847f-e3ec-4647-aacb-f3e382419dda None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:49.947268 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172fa0c-4fab-4142-ae45-74f4436de844 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:49.969446 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d46dce-18cc-4bab-998b-fbcc5c670f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:49.970145 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d46dce-18cc-4bab-998b-fbcc5c670f8c None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:49.970437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d46dce-18cc-4bab-998b-fbcc5c670f8c None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:49.970722 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d46dce-18cc-4bab-998b-fbcc5c670f8c None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:49.971473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d46dce-18cc-4bab-998b-fbcc5c670f8c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:49.971785 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d46dce-18cc-4bab-998b-fbcc5c670f8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:49.973158 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d46dce-18cc-4bab-998b-fbcc5c670f8c None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:49.985840 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 516/1058] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Jun 27 15:35:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:49.986029 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91168d-c611-405d-a937-326add69354b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:49.986128 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d46dce-18cc-4bab-998b-fbcc5c670f8c None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:49.986222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91168d-c611-405d-a937-326add69354b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:49.986222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91168d-c611-405d-a937-326add69354b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:49.986222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91168d-c611-405d-a937-326add69354b None admin] PATH_INFO: `/v3/services/image` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:49.986222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91168d-c611-405d-a937-326add69354b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:49.986222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91168d-c611-405d-a937-326add69354b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:49.997078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-bc91168d-c611-405d-a937-326add69354b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 15:35:49.997078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:49.997078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:35:49.997078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:35:49.997078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:35:49.997078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:35:49.997078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:35:49.997078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:49.997078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 15:35:49.997078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:49.997078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:35:49.997078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:49.997078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:49.997078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:49.997078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:49.997078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:49.997078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:49.998254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:49.998254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:49.998254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:49.998254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:49.998254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:49.998254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:49.998254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:35:49.998254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:35:49.998254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:35:49.998254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:35:49.998254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:49.998254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:49.998254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:49.998254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:49.998254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:49.998254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:49.999112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:49.999112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:49.999112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:49.999112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:49.999112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:49.999112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:49.999112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:49.999112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:49.999112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:49.999112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:49.999112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:35:49.999112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:35:49.999112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 15:35:49.999112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:49.999112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 543/1059] 10.222.0.20 () {56 vars in 1105 bytes} [Mon Jun 27 15:35:49 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:49.999912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91168d-c611-405d-a937-326add69354b None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:50.050301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:50.064106 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:50.091945 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d46dce-18cc-4bab-998b-fbcc5c670f8c None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:50.103739 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91168d-c611-405d-a937-326add69354b None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:50.130779 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc0800-b953-4e43-ba4e-2c4174a84992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:50.130779 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc0800-b953-4e43-ba4e-2c4174a84992 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:50.131094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc0800-b953-4e43-ba4e-2c4174a84992 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:50.131258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc0800-b953-4e43-ba4e-2c4174a84992 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:50.132095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc0800-b953-4e43-ba4e-2c4174a84992 None admin] RBAC: Adding query filter params (role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.project.id=6fddfead5e6c40e289cd1973ec85c380, user.id=bb6dd937abbd432e946d06713440c7d1) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:50.132406 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc0800-b953-4e43-ba4e-2c4174a84992 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:50.136766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc0800-b953-4e43-ba4e-2c4174a84992 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:50.140845 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dbe63a-3ef4-46d9-b92a-00f63a9cc5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:50.141659 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbe63a-3ef4-46d9-b92a-00f63a9cc5e4 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:50.141952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbe63a-3ef4-46d9-b92a-00f63a9cc5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:50.142229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbe63a-3ef4-46d9-b92a-00f63a9cc5e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:50.144328 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 517/1060] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:35:49 2022] GET /identity/v3/role_assignments?user.id=bb6dd937abbd432e946d06713440c7d1&scope.project.id=6fddfead5e6c40e289cd1973ec85c380&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:50.145169 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc0800-b953-4e43-ba4e-2c4174a84992 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:50.157694 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dbe63a-3ef4-46d9-b92a-00f63a9cc5e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:50.159168 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dbe63a-3ef4-46d9-b92a-00f63a9cc5e4 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:50.179384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-72dbe63a-3ef4-46d9-b92a-00f63a9cc5e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:50.179384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:50.179384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:50.179384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:50.179384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:50.179384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:50.179384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:50.179384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:50.179384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:50.179384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:50.179384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:50.179384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:50.179384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:50.179384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:50.179384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:50.180539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:50.180539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:50.180539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:50.180539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:50.180539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:50.180539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:50.180539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:50.180539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:50.180539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:50.180539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:50.180539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:50.180539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:50.180539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:50.180539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:50.180539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:50.180539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:50.180539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:50.181620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:50.181620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:50.181620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:50.181620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:50.181620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 544/1061] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:50.186700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dbe63a-3ef4-46d9-b92a-00f63a9cc5e4 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:50.281457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc0800-b953-4e43-ba4e-2c4174a84992 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:50.314519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dbe63a-3ef4-46d9-b92a-00f63a9cc5e4 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:50.315796 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:50.324287 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600af920-b53d-40fd-9819-8939be94ef91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:50.324982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600af920-b53d-40fd-9819-8939be94ef91 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:50.325267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600af920-b53d-40fd-9819-8939be94ef91 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:50.325543 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600af920-b53d-40fd-9819-8939be94ef91 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:50.326321 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600af920-b53d-40fd-9819-8939be94ef91 None admin] RBAC: Adding query filter params (name=image) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:50.326633 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600af920-b53d-40fd-9819-8939be94ef91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:50.328114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600af920-b53d-40fd-9819-8939be94ef91 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:50.333361 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 518/1062] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:50.339570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600af920-b53d-40fd-9819-8939be94ef91 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:50.342381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:50.361362 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b995e-90a1-4132-aa48-c2740920d295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:50.362142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b995e-90a1-4132-aa48-c2740920d295 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:50.362430 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b995e-90a1-4132-aa48-c2740920d295 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:50.362710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b995e-90a1-4132-aa48-c2740920d295 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:50.363441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b995e-90a1-4132-aa48-c2740920d295 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:50.363851 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b995e-90a1-4132-aa48-c2740920d295 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:50.365240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b995e-90a1-4132-aa48-c2740920d295 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:50.378360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 545/1063] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:50.384515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b995e-90a1-4132-aa48-c2740920d295 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:50.457153 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600af920-b53d-40fd-9819-8939be94ef91 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:50.496212 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b995e-90a1-4132-aa48-c2740920d295 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:50.498629 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2d0ae-b0fa-4912-b1b7-8da47df42592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:50.499315 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2d0ae-b0fa-4912-b1b7-8da47df42592 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:50.499597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2d0ae-b0fa-4912-b1b7-8da47df42592 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:50.499872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2d0ae-b0fa-4912-b1b7-8da47df42592 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:50.500596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2d0ae-b0fa-4912-b1b7-8da47df42592 None admin] RBAC: Adding query filter params (type=image) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:50.500936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2d0ae-b0fa-4912-b1b7-8da47df42592 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:50.502368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2d0ae-b0fa-4912-b1b7-8da47df42592 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:50.508031 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 519/1064] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:50.513983 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2d0ae-b0fa-4912-b1b7-8da47df42592 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:50.542859 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b897d-c4d6-4711-8262-1e15dadabb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:50.544158 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b897d-c4d6-4711-8262-1e15dadabb7a None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:50.544517 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b897d-c4d6-4711-8262-1e15dadabb7a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:50.544815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b897d-c4d6-4711-8262-1e15dadabb7a None admin] PATH_INFO: `/v3/users/nova` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:50.553159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b897d-c4d6-4711-8262-1e15dadabb7a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:50.554651 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b897d-c4d6-4711-8262-1e15dadabb7a None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:50.570579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-e40b897d-c4d6-4711-8262-1e15dadabb7a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:35:50.570579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:50.570579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:50.570579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:50.570579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:50.570579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:50.570579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:50.570579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:50.570579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:50.570579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:50.570579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:50.570579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:50.570579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:35:50.570579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:35:50.570579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:35:50.570579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:35:50.571890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:50.571890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:50.571890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:50.571890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:50.571890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:50.571890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:50.571890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:50.571890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:50.571890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:50.571890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:50.571890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:50.571890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:50.571890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:50.571890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:50.571890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:50.571890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:50.571890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:50.571890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:50.573052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:50.573052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:50.573052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:35:50.573052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:35:50.573052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:35:50.573052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:35:50.573052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:35:50.573052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:35:50.573052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:35:50.573052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:50.573052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 546/1065] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Jun 27 15:35:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:50.578232 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b897d-c4d6-4711-8262-1e15dadabb7a None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:50.600251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:50.609602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:50.640033 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2d0ae-b0fa-4912-b1b7-8da47df42592 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:50.682776 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f455a-935c-4ff9-92c9-4c15c0b272de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:50.683472 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f455a-935c-4ff9-92c9-4c15c0b272de None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:50.683757 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f455a-935c-4ff9-92c9-4c15c0b272de None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:50.684030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f455a-935c-4ff9-92c9-4c15c0b272de None admin] PATH_INFO: `/v3/endpoints` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:50.684844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7f455a-935c-4ff9-92c9-4c15c0b272de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:50.686351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7f455a-935c-4ff9-92c9-4c15c0b272de None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:50.702559 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b897d-c4d6-4711-8262-1e15dadabb7a None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:50.717676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 520/1066] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Jun 27 15:35:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:35:50.750550 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b19d17-adbe-42b1-9340-ece441a82194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:50.751248 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b19d17-adbe-42b1-9340-ece441a82194 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:50.751535 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b19d17-adbe-42b1-9340-ece441a82194 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:50.751814 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b19d17-adbe-42b1-9340-ece441a82194 None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:50.752688 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b19d17-adbe-42b1-9340-ece441a82194 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:50.753000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b19d17-adbe-42b1-9340-ece441a82194 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:50.754507 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b19d17-adbe-42b1-9340-ece441a82194 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:50.779863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 547/1067] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Jun 27 15:35:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:50.785754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f455a-935c-4ff9-92c9-4c15c0b272de None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:50.909461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f455a-935c-4ff9-92c9-4c15c0b272de None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:50.937814 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:50.952469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16182bc0-18d7-4729-9ff5-8e46ca3fe222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:50.953277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16182bc0-18d7-4729-9ff5-8e46ca3fe222 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:50.953562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16182bc0-18d7-4729-9ff5-8e46ca3fe222 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:50.953865 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16182bc0-18d7-4729-9ff5-8e46ca3fe222 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:50.966442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16182bc0-18d7-4729-9ff5-8e46ca3fe222 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:50.967912 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16182bc0-18d7-4729-9ff5-8e46ca3fe222 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:50.986810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-16182bc0-18d7-4729-9ff5-8e46ca3fe222 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:50.986810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:50.986810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:50.986810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:50.986810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:50.986810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:50.986810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:50.986810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:50.986810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:50.986810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:50.986810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:50.986810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:50.986810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:50.986810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:50.986810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:50.987802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:50.987802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:50.987802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:50.987802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:50.987802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:50.987802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:50.987802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:50.987802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:50.987802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:50.987802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:50.987802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:50.987802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:50.987802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:50.987802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:50.987802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:50.987802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:50.987802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:50.989067 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:50.989067 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:50.989067 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:50.989067 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:50.989067 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 521/1068] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:50.993948 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b19d17-adbe-42b1-9340-ece441a82194 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:51.075201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:51.100628 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b19d17-adbe-42b1-9340-ece441a82194 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:51.135934 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77184e1-49be-4b28-aac3-e4551ca111b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:51.136606 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77184e1-49be-4b28-aac3-e4551ca111b5 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:51.136890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77184e1-49be-4b28-aac3-e4551ca111b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:51.137168 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77184e1-49be-4b28-aac3-e4551ca111b5 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:51.137924 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77184e1-49be-4b28-aac3-e4551ca111b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:51.138237 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77184e1-49be-4b28-aac3-e4551ca111b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:51.139617 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77184e1-49be-4b28-aac3-e4551ca111b5 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:51.156791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 548/1069] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:51.158196 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16182bc0-18d7-4729-9ff5-8e46ca3fe222 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:51.269127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16182bc0-18d7-4729-9ff5-8e46ca3fe222 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:51.308180 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb06e8-dc43-419d-ba7c-35adfe22315c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:51.309003 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb06e8-dc43-419d-ba7c-35adfe22315c None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:51.309288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb06e8-dc43-419d-ba7c-35adfe22315c None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:51.309578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb06e8-dc43-419d-ba7c-35adfe22315c None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:51.316963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb06e8-dc43-419d-ba7c-35adfe22315c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:51.318459 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb06e8-dc43-419d-ba7c-35adfe22315c None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:51.333819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-fadb06e8-dc43-419d-ba7c-35adfe22315c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:51.333819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:51.333819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:51.333819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:51.333819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:51.333819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:51.333819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:51.333819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:51.333819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:51.333819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:51.333819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:51.333819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:51.333819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:51.333819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:51.333819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:51.334744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:51.334744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:51.334744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:51.334744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:51.334744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:51.334744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:51.334744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:51.334744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:51.334744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:51.334744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:51.334744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:51.334744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:51.334744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:51.334744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:51.334744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:51.334744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:51.334744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:51.335636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:51.335636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:51.335636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:51.335636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:51.335636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:51.335636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:51.335636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:51.335636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:51.335636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 522/1070] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:51.341027 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77184e1-49be-4b28-aac3-e4551ca111b5 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:51.447500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77184e1-49be-4b28-aac3-e4551ca111b5 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:51.482073 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf27806-3940-4bee-8978-1220b8d688ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:51.482795 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf27806-3940-4bee-8978-1220b8d688ef None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:51.483085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf27806-3940-4bee-8978-1220b8d688ef None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:51.483361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf27806-3940-4bee-8978-1220b8d688ef None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:51.484155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf27806-3940-4bee-8978-1220b8d688ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:51.484466 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf27806-3940-4bee-8978-1220b8d688ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:51.485914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf27806-3940-4bee-8978-1220b8d688ef None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:51.488614 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:51.505341 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 549/1071] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:51.506536 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb06e8-dc43-419d-ba7c-35adfe22315c None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:51.546941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:51.623316 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb06e8-dc43-419d-ba7c-35adfe22315c None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:51.658611 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e034cd0-4ed3-4d1b-88fe-88c4fb4e293a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:51.659565 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e034cd0-4ed3-4d1b-88fe-88c4fb4e293a None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:51.659858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e034cd0-4ed3-4d1b-88fe-88c4fb4e293a None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:51.660138 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e034cd0-4ed3-4d1b-88fe-88c4fb4e293a None admin] PATH_INFO: `/v3/projects/7be83fb6f20341139147373837320148/users/1c8f6262cf24419daf803ae71aed24f5/roles/507fb7d64e4549c58ff859abf6160c43` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:51.662373 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e034cd0-4ed3-4d1b-88fe-88c4fb4e293a None admin] RBAC: Authorizing `identity:create_grant(project_id=7be83fb6f20341139147373837320148, user_id=1c8f6262cf24419daf803ae71aed24f5, role_id=507fb7d64e4549c58ff859abf6160c43)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:51.663818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e034cd0-4ed3-4d1b-88fe-88c4fb4e293a None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:51.685679 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 523/1072] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:35:51 2022] PUT /identity/v3/projects/7be83fb6f20341139147373837320148/users/1c8f6262cf24419daf803ae71aed24f5/roles/507fb7d64e4549c58ff859abf6160c43 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:35:52.254308 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833321e-7336-413b-b44c-c9fe63c32334 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:52.254831 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833321e-7336-413b-b44c-c9fe63c32334 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:52.254939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833321e-7336-413b-b44c-c9fe63c32334 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:52.256375 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 550/1073] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:52.262279 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6cd6e-885b-4126-9914-ffd064b0ed93 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:52.262692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6cd6e-885b-4126-9914-ffd064b0ed93 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:52.262862 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6cd6e-885b-4126-9914-ffd064b0ed93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:52.281429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-e5d6cd6e-885b-4126-9914-ffd064b0ed93 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:52.282672 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e5d6cd6e-885b-4126-9914-ffd064b0ed93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:52.357260 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 524/1074] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:52.380402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833321e-7336-413b-b44c-c9fe63c32334 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:52.496405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8833321e-7336-413b-b44c-c9fe63c32334 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:52.536195 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430791de-a2c2-4382-a5c1-a5e9fd7f3a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:52.536914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430791de-a2c2-4382-a5c1-a5e9fd7f3a34 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:52.537198 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430791de-a2c2-4382-a5c1-a5e9fd7f3a34 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:52.537478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430791de-a2c2-4382-a5c1-a5e9fd7f3a34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:52.549165 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430791de-a2c2-4382-a5c1-a5e9fd7f3a34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:52.557486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-430791de-a2c2-4382-a5c1-a5e9fd7f3a34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:35:52.557486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:52.557486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:52.557486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:52.557486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:52.557486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:52.557486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:52.557486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:52.557486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:52.557486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:52.557486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:52.557486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:52.557486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:52.557486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:52.557486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:52.557486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:52.558437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:52.558437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:52.558437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:52.558437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:52.558437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:52.558437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:52.558437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:52.558437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:52.558437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:52.558437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:52.558437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:52.558437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:52.558437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:52.558437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:52.558437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:52.558437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:52.559281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:52.559281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:52.559281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:52.559281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:52.559281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:52.559281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:52.559281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:52.559281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:52.559281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:52.559281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:52.559281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:35:52.559281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:52.561972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 551/1075] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:35:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:52.564809 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6cd6e-885b-4126-9914-ffd064b0ed93 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:52.590826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:52.672055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6cd6e-885b-4126-9914-ffd064b0ed93 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:52.707615 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837d7a1-743f-44e1-856c-38f2367631d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:52.708291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837d7a1-743f-44e1-856c-38f2367631d4 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:52.708578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837d7a1-743f-44e1-856c-38f2367631d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:52.708856 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837d7a1-743f-44e1-856c-38f2367631d4 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:52.709606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837d7a1-743f-44e1-856c-38f2367631d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:52.709939 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837d7a1-743f-44e1-856c-38f2367631d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:52.711390 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837d7a1-743f-44e1-856c-38f2367631d4 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:52.724481 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 525/1076] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:35:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:52.725694 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430791de-a2c2-4382-a5c1-a5e9fd7f3a34 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:52.785177 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:52.847773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430791de-a2c2-4382-a5c1-a5e9fd7f3a34 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:52.882316 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96646c16-f389-4d32-bf7e-39083c8fa83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:52.883744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96646c16-f389-4d32-bf7e-39083c8fa83f None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:52.884025 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96646c16-f389-4d32-bf7e-39083c8fa83f None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:52.884299 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96646c16-f389-4d32-bf7e-39083c8fa83f None admin] PATH_INFO: `/v3/users/bb6dd937abbd432e946d06713440c7d1` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:52.885506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96646c16-f389-4d32-bf7e-39083c8fa83f None admin] RBAC: Authorizing `identity:get_user(user_id=bb6dd937abbd432e946d06713440c7d1)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:52.887005 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96646c16-f389-4d32-bf7e-39083c8fa83f None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:52.893229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 552/1077] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:35:52 2022] GET /identity/v3/users/bb6dd937abbd432e946d06713440c7d1 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:52.894430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837d7a1-743f-44e1-856c-38f2367631d4 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:52.910203 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f45299-c150-471c-909c-51aaeb50cbe3 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:52.910519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f45299-c150-471c-909c-51aaeb50cbe3 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:52.910770 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f45299-c150-471c-909c-51aaeb50cbe3 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:52.912290 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 553/1078] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:52.918494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127caa3f-f924-4133-b35f-54dd120f03aa None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:52.918783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127caa3f-f924-4133-b35f-54dd120f03aa None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:52.919070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127caa3f-f924-4133-b35f-54dd120f03aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:52.938151 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-127caa3f-f924-4133-b35f-54dd120f03aa None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:52.939638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-127caa3f-f924-4133-b35f-54dd120f03aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:52.993411 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:53.012997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837d7a1-743f-44e1-856c-38f2367631d4 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:53.020919 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 554/1079] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:35:53.044801 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127caa3f-f924-4133-b35f-54dd120f03aa None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:53.051767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0521c54-4d88-4593-9ce0-5000588e2cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:53.052693 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0521c54-4d88-4593-9ce0-5000588e2cfc None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:53.052975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0521c54-4d88-4593-9ce0-5000588e2cfc None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:53.053284 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0521c54-4d88-4593-9ce0-5000588e2cfc None admin] PATH_INFO: `/v3/projects/6fddfead5e6c40e289cd1973ec85c380` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:53.054543 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0521c54-4d88-4593-9ce0-5000588e2cfc None admin] RBAC: Authorizing `identity:get_project(project_id=6fddfead5e6c40e289cd1973ec85c380)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:53.056118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0521c54-4d88-4593-9ce0-5000588e2cfc None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:53.057963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 526/1080] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:35:52 2022] GET /identity/v3/projects/6fddfead5e6c40e289cd1973ec85c380 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:53.063635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0521c54-4d88-4593-9ce0-5000588e2cfc None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:53.108222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:53.154348 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127caa3f-f924-4133-b35f-54dd120f03aa None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:53.176015 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0521c54-4d88-4593-9ce0-5000588e2cfc None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:53.196728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adafec9a-cc70-4e27-9001-4f72509830b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:53.197766 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafec9a-cc70-4e27-9001-4f72509830b5 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:53.198211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafec9a-cc70-4e27-9001-4f72509830b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:53.198649 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafec9a-cc70-4e27-9001-4f72509830b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:53.216252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be106888-dbe9-4024-ad9b-8c66963851c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:53.217332 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be106888-dbe9-4024-ad9b-8c66963851c7 None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:53.217660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be106888-dbe9-4024-ad9b-8c66963851c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:53.217954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be106888-dbe9-4024-ad9b-8c66963851c7 None admin] PATH_INFO: `/v3/projects/6fddfead5e6c40e289cd1973ec85c380/users/bb6dd937abbd432e946d06713440c7d1/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:53.220008 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be106888-dbe9-4024-ad9b-8c66963851c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=6fddfead5e6c40e289cd1973ec85c380, user_id=bb6dd937abbd432e946d06713440c7d1, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:53.221507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be106888-dbe9-4024-ad9b-8c66963851c7 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:53.235803 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 527/1081] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:35:53 2022] PUT /identity/v3/projects/6fddfead5e6c40e289cd1973ec85c380/users/bb6dd937abbd432e946d06713440c7d1/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:35:53.287314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adafec9a-cc70-4e27-9001-4f72509830b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:53.288824 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adafec9a-cc70-4e27-9001-4f72509830b5 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:53.353338 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:53.408522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 555/1082] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:35:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 15:35:53.415978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be106888-dbe9-4024-ad9b-8c66963851c7 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:53.435966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:53.526162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be106888-dbe9-4024-ad9b-8c66963851c7 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:53.561698 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b24e29-1f0f-4953-a31e-0ab88f19a499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:53.562501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b24e29-1f0f-4953-a31e-0ab88f19a499 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:53.562785 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b24e29-1f0f-4953-a31e-0ab88f19a499 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:53.563061 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b24e29-1f0f-4953-a31e-0ab88f19a499 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:53.574854 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b24e29-1f0f-4953-a31e-0ab88f19a499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:53.576318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b24e29-1f0f-4953-a31e-0ab88f19a499 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:53.593863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-a5b24e29-1f0f-4953-a31e-0ab88f19a499 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:53.593863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:53.593863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:53.593863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:53.593863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:53.593863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:53.593863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:53.593863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:53.593863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:53.593863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:53.593863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:53.593863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:53.593863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:53.593863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:53.593863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:53.594888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:53.594888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:53.594888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:53.594888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:53.594888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:53.594888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:53.594888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:53.594888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:53.594888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:53.594888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:53.594888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:53.594888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:53.594888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:53.594888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:53.594888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:53.594888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:53.594888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:53.595867 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:53.595867 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:53.595867 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:53.595867 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:53.595867 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 528/1083] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:53.601436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adafec9a-cc70-4e27-9001-4f72509830b5 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:53.703067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adafec9a-cc70-4e27-9001-4f72509830b5 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:53.739861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b03c4-c982-4f10-8e72-8ffff8db5da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:53.740530 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b03c4-c982-4f10-8e72-8ffff8db5da7 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:53.740811 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b03c4-c982-4f10-8e72-8ffff8db5da7 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:53.741083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b03c4-c982-4f10-8e72-8ffff8db5da7 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:53.741834 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b03c4-c982-4f10-8e72-8ffff8db5da7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:53.742143 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b03c4-c982-4f10-8e72-8ffff8db5da7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:53.744790 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b03c4-c982-4f10-8e72-8ffff8db5da7 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:53.757985 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 556/1084] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:53.763311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12e377-6225-429c-b61a-ad0e8ce4c3b7 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:53.763603 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12e377-6225-429c-b61a-ad0e8ce4c3b7 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:53.763885 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12e377-6225-429c-b61a-ad0e8ce4c3b7 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:53.765327 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 529/1085] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:53.771411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee2002-95d2-46fd-8fc3-b0dcb5d26abd None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:53.771699 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee2002-95d2-46fd-8fc3-b0dcb5d26abd None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:53.771981 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee2002-95d2-46fd-8fc3-b0dcb5d26abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:53.791377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-51ee2002-95d2-46fd-8fc3-b0dcb5d26abd None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:53.792614 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-51ee2002-95d2-46fd-8fc3-b0dcb5d26abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:53.804458 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:53.886243 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 557/1086] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:35:53.915845 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12e377-6225-429c-b61a-ad0e8ce4c3b7 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:53.998295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:54.044738 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12e377-6225-429c-b61a-ad0e8ce4c3b7 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:54.086503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73ec7c-3d9d-47c7-8a62-4d1e7abe4f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:54.087496 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73ec7c-3d9d-47c7-8a62-4d1e7abe4f6b None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:54.087837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73ec7c-3d9d-47c7-8a62-4d1e7abe4f6b None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:54.088174 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73ec7c-3d9d-47c7-8a62-4d1e7abe4f6b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:54.100489 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73ec7c-3d9d-47c7-8a62-4d1e7abe4f6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:54.111029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-7b73ec7c-3d9d-47c7-8a62-4d1e7abe4f6b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:35:54.111029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:54.111029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:54.111029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:54.111029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:54.111029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:54.111029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:54.111029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:54.111029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:54.111029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:54.111029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:54.111029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:54.111029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:54.111029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:54.111029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:54.111029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:54.112154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:54.112154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:54.112154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:54.112154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:54.112154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:54.112154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:54.112154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:54.112154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:54.112154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:54.112154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:54.112154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:54.112154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:54.112154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:54.112154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:54.112154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:54.112154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:54.113295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:54.113295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:54.113295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:54.113295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:54.113295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:54.113295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:54.113295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:54.113295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:54.113295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:54.113295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:54.113295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 15:35:54.113295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:54.113295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 530/1087] 10.222.0.20 () {56 vars in 1120 bytes} [Mon Jun 27 15:35:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:54.121230 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee2002-95d2-46fd-8fc3-b0dcb5d26abd None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:54.219067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee2002-95d2-46fd-8fc3-b0dcb5d26abd None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:54.253995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aebb29-f914-4f13-9e87-1c6f63c88b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:54.254591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aebb29-f914-4f13-9e87-1c6f63c88b36 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:54.254878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aebb29-f914-4f13-9e87-1c6f63c88b36 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:54.255162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aebb29-f914-4f13-9e87-1c6f63c88b36 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:54.255914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aebb29-f914-4f13-9e87-1c6f63c88b36 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:54.256223 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aebb29-f914-4f13-9e87-1c6f63c88b36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:54.257602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aebb29-f914-4f13-9e87-1c6f63c88b36 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:54.265776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 558/1088] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Jun 27 15:35:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:54.271443 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73ec7c-3d9d-47c7-8a62-4d1e7abe4f6b None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:54.380482 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73ec7c-3d9d-47c7-8a62-4d1e7abe4f6b None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:54.408733 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:54.418531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f795e63-5407-47e3-9af8-d7ac62aca4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:54.419365 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f795e63-5407-47e3-9af8-d7ac62aca4b4 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:54.419648 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f795e63-5407-47e3-9af8-d7ac62aca4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:54.419924 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f795e63-5407-47e3-9af8-d7ac62aca4b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:54.430982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f795e63-5407-47e3-9af8-d7ac62aca4b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:54.432449 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f795e63-5407-47e3-9af8-d7ac62aca4b4 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:54.450200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-1f795e63-5407-47e3-9af8-d7ac62aca4b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:54.450200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:54.450200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:54.450200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:54.450200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:54.450200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:54.450200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:54.450200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:54.450200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:54.450200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:54.450200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:54.450200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:54.450200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:54.450200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:54.450200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:54.451307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:54.451307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:54.451307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:54.451307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:54.451307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:54.451307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:54.451307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:54.451307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:54.451307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:54.451307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:54.451307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:54.451307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:54.451307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:54.451307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:54.451307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:54.451307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:54.451307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:54.452218 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:54.452218 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:54.452218 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:54.452218 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:54.452218 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 531/1089] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:54.457134 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aebb29-f914-4f13-9e87-1c6f63c88b36 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:54.467382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:54.562795 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aebb29-f914-4f13-9e87-1c6f63c88b36 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:54.596251 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fbacc-8017-4abe-8cac-2a7043707149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:54.596913 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fbacc-8017-4abe-8cac-2a7043707149 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:54.597222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fbacc-8017-4abe-8cac-2a7043707149 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:54.597498 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fbacc-8017-4abe-8cac-2a7043707149 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:54.598272 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fbacc-8017-4abe-8cac-2a7043707149 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:54.598581 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fbacc-8017-4abe-8cac-2a7043707149 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:54.599951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fbacc-8017-4abe-8cac-2a7043707149 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:54.615204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 559/1090] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:54.622182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f795e63-5407-47e3-9af8-d7ac62aca4b4 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:54.723615 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f795e63-5407-47e3-9af8-d7ac62aca4b4 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:54.758923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c569d54-63c0-41df-9e86-6053cf884369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:54.759721 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c569d54-63c0-41df-9e86-6053cf884369 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:54.760005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c569d54-63c0-41df-9e86-6053cf884369 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:54.760280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c569d54-63c0-41df-9e86-6053cf884369 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:54.768193 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c569d54-63c0-41df-9e86-6053cf884369 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:54.769660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c569d54-63c0-41df-9e86-6053cf884369 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:54.784025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-6c569d54-63c0-41df-9e86-6053cf884369 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:35:54.784025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:54.784025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:54.784025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:54.784025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:54.784025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:54.784025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:54.784025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:54.784025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:54.784025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:54.784025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:54.784025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:54.784025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:35:54.784025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:35:54.784025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:35:54.784025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:35:54.785080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:54.785080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:54.785080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:35:54.785080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:54.785080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:35:54.785080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:35:54.785080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:54.785080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:54.785080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:54.785080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:54.785080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:54.785080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:54.785080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:54.785080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:54.785080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:54.785080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:54.785080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:54.785080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:54.786118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:54.786118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:54.786118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:35:54.786118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:35:54.786118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:35:54.786118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:35:54.786118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:35:54.786118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:35:54.786118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 15:35:54.786118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:54.786118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 532/1091] 10.222.0.20 () {56 vars in 1093 bytes} [Mon Jun 27 15:35:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:54.791487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fbacc-8017-4abe-8cac-2a7043707149 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:54.867208 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:54.908686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fbacc-8017-4abe-8cac-2a7043707149 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:54.949354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0e8e8-8d82-47a0-8be7-6111a735d725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:54.950117 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0e8e8-8d82-47a0-8be7-6111a735d725 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:54.950464 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0e8e8-8d82-47a0-8be7-6111a735d725 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:54.950799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0e8e8-8d82-47a0-8be7-6111a735d725 None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:54.953836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0e8e8-8d82-47a0-8be7-6111a735d725 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:54.953836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0e8e8-8d82-47a0-8be7-6111a735d725 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:54.954061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0e8e8-8d82-47a0-8be7-6111a735d725 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:54.975326 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 560/1092] 10.222.0.20 () {56 vars in 1133 bytes} [Mon Jun 27 15:35:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:54.982039 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c569d54-63c0-41df-9e86-6053cf884369 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:55.024894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:55.088902 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c569d54-63c0-41df-9e86-6053cf884369 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:55.126457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73c9aa-d903-44ab-95fb-d357a33bdef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:55.127116 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73c9aa-d903-44ab-95fb-d357a33bdef3 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:55.127406 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73c9aa-d903-44ab-95fb-d357a33bdef3 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:55.127686 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73c9aa-d903-44ab-95fb-d357a33bdef3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:55.199592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73c9aa-d903-44ab-95fb-d357a33bdef3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:55.201042 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73c9aa-d903-44ab-95fb-d357a33bdef3 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:55.239841 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:55.303693 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 533/1093] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:35:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:35:55.310039 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0e8e8-8d82-47a0-8be7-6111a735d725 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:55.333125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b5d8d-a698-4499-8d17-565ced0ddc0a None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:55.333410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b5d8d-a698-4499-8d17-565ced0ddc0a None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:55.333740 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b5d8d-a698-4499-8d17-565ced0ddc0a None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:55.335133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 534/1094] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:35:55.341334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a4546-088b-491f-8715-6de910cf2b03 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:55.341558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a4546-088b-491f-8715-6de910cf2b03 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:55.341870 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a4546-088b-491f-8715-6de910cf2b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:55.363273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-4d0a4546-088b-491f-8715-6de910cf2b03 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:55.364494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4d0a4546-088b-491f-8715-6de910cf2b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:55.420868 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0e8e8-8d82-47a0-8be7-6111a735d725 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:55.437431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 535/1095] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:35:55.445438 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:55.456819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3c94f-4bc1-47a1-a6f9-fffc94b0b3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:55.457685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3c94f-4bc1-47a1-a6f9-fffc94b0b3da None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:55.458199 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3c94f-4bc1-47a1-a6f9-fffc94b0b3da None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:55.458294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3c94f-4bc1-47a1-a6f9-fffc94b0b3da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:55.462603 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a4546-088b-491f-8715-6de910cf2b03 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:55.469546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3c94f-4bc1-47a1-a6f9-fffc94b0b3da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:55.471111 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3c94f-4bc1-47a1-a6f9-fffc94b0b3da None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:55.489980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-7ca3c94f-4bc1-47a1-a6f9-fffc94b0b3da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:55.489980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:55.489980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:55.489980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:55.489980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:55.489980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:55.489980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:55.489980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:55.489980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:55.489980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:55.489980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:55.489980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:55.489980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:35:55.489980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:35:55.489980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:35:55.490945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:35:55.490945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:55.490945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:55.490945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:55.490945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:55.490945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:55.490945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:55.490945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:55.490945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:55.490945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:55.490945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:55.490945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:55.490945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:55.490945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:55.490945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:55.490945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:55.490945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:55.491802 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:35:55.491802 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:35:55.491802 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:35:55.491802 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:55.491802 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 561/1096] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:35:55.498255 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3c94f-4bc1-47a1-a6f9-fffc94b0b3da None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:55.538356 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:55.575570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a4546-088b-491f-8715-6de910cf2b03 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:55.621400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3c94f-4bc1-47a1-a6f9-fffc94b0b3da None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:55.622881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b54b49-a9f4-4dc7-b2ee-65c02393a992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:55.623701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b54b49-a9f4-4dc7-b2ee-65c02393a992 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:55.623984 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b54b49-a9f4-4dc7-b2ee-65c02393a992 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:55.624264 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b54b49-a9f4-4dc7-b2ee-65c02393a992 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:55.637211 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b54b49-a9f4-4dc7-b2ee-65c02393a992 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:35:55.646088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-64b54b49-a9f4-4dc7-b2ee-65c02393a992 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:35:55.646088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:55.646088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:55.646088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:55.646088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:55.646088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:55.646088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:55.646088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:55.646088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:55.646088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:55.646088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:55.646088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:55.646088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:35:55.646088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:35:55.646088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:35:55.646088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:35:55.647154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:35:55.647154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:35:55.647154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:35:55.647154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:35:55.647154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:55.647154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:55.647154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:55.647154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:55.647154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:55.647154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:55.647154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:55.647154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:55.647154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:55.647154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:55.647154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:55.647154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:55.648005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:55.648005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:55.648005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:55.648005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:55.648005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:35:55.648005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:35:55.648005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:35:55.648005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:35:55.648005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:35:55.648005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:35:55.648005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:35:55.648005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:35:55.648005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 536/1097] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:35:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:55.654536 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b54b49-a9f4-4dc7-b2ee-65c02393a992 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:55.667071 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7d802-af1d-4b65-a959-6232d800fb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:55.667772 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7d802-af1d-4b65-a959-6232d800fb57 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:55.668063 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7d802-af1d-4b65-a959-6232d800fb57 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:55.668341 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7d802-af1d-4b65-a959-6232d800fb57 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:55.670949 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7d802-af1d-4b65-a959-6232d800fb57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:55.671301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7d802-af1d-4b65-a959-6232d800fb57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:55.672729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7d802-af1d-4b65-a959-6232d800fb57 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:55.685720 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 562/1098] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:55.692089 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7d802-af1d-4b65-a959-6232d800fb57 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:55.735020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:55.778110 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b54b49-a9f4-4dc7-b2ee-65c02393a992 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:55.809520 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:55.818816 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b020e2-a790-4c14-8c61-00121ea5fb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:55.819549 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b020e2-a790-4c14-8c61-00121ea5fb61 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:55.819873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7d802-af1d-4b65-a959-6232d800fb57 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:55.819982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b020e2-a790-4c14-8c61-00121ea5fb61 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:55.820160 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b020e2-a790-4c14-8c61-00121ea5fb61 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:55.821607 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b020e2-a790-4c14-8c61-00121ea5fb61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:55.822137 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b020e2-a790-4c14-8c61-00121ea5fb61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:55.823729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b020e2-a790-4c14-8c61-00121ea5fb61 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:55.833399 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 537/1099] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:35:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:55.842703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b020e2-a790-4c14-8c61-00121ea5fb61 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:55.863478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad6f2a-a6a2-4b72-8e31-76e7b2e74f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:55.864372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad6f2a-a6a2-4b72-8e31-76e7b2e74f6d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:55.864664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad6f2a-a6a2-4b72-8e31-76e7b2e74f6d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:55.864984 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad6f2a-a6a2-4b72-8e31-76e7b2e74f6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:55.873229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad6f2a-a6a2-4b72-8e31-76e7b2e74f6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:55.874918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad6f2a-a6a2-4b72-8e31-76e7b2e74f6d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:55.891805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-08ad6f2a-a6a2-4b72-8e31-76e7b2e74f6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:55.891805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:35:55.891805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:35:55.891805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:35:55.891805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:35:55.891805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:35:55.891805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:35:55.891805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:35:55.891805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:35:55.891805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:35:55.891805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:35:55.891805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:35:55.891805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:35:55.891805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:35:55.891805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:35:55.893033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:35:55.893033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:35:55.893033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:35:55.893033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:35:55.893033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:35:55.893033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:35:55.893033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:35:55.893033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:35:55.893033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:35:55.893033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:35:55.893033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:35:55.893033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:35:55.893033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:35:55.893033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:35:55.893033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:35:55.893033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:35:55.893033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:35:55.894094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:35:55.894094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:35:55.894094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:35:55.894094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:35:55.894094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:35:55.894094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:35:55.894094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:35:55.894094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:35:55.894094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 563/1100] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:35:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:35:55.900869 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad6f2a-a6a2-4b72-8e31-76e7b2e74f6d None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:55.970975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b020e2-a790-4c14-8c61-00121ea5fb61 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:56.011639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711912d-0b6d-42b2-b635-ad1431df1b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:56.012527 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711912d-0b6d-42b2-b635-ad1431df1b16 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:56.012809 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711912d-0b6d-42b2-b635-ad1431df1b16 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:56.013086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711912d-0b6d-42b2-b635-ad1431df1b16 None admin] PATH_INFO: `/v3/users/bb6dd937abbd432e946d06713440c7d1` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:56.014349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8711912d-0b6d-42b2-b635-ad1431df1b16 None admin] RBAC: Authorizing `identity:get_user(user_id=bb6dd937abbd432e946d06713440c7d1)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:56.015803 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8711912d-0b6d-42b2-b635-ad1431df1b16 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:56.017493 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 538/1101] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:35:55 2022] GET /identity/v3/users/bb6dd937abbd432e946d06713440c7d1 => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:56.020450 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:56.025959 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711912d-0b6d-42b2-b635-ad1431df1b16 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:56.028820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad6f2a-a6a2-4b72-8e31-76e7b2e74f6d None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:56.071879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd233eb1-48ef-42a3-952b-45d75f61c93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:56.072720 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd233eb1-48ef-42a3-952b-45d75f61c93d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:56.073007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd233eb1-48ef-42a3-952b-45d75f61c93d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:56.073421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd233eb1-48ef-42a3-952b-45d75f61c93d None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:56.074667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd233eb1-48ef-42a3-952b-45d75f61c93d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:56.074982 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd233eb1-48ef-42a3-952b-45d75f61c93d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:56.076424 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd233eb1-48ef-42a3-952b-45d75f61c93d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:56.096506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 564/1102] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:35:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:56.097569 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:56.098014 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd233eb1-48ef-42a3-952b-45d75f61c93d None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:56.147936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711912d-0b6d-42b2-b635-ad1431df1b16 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:56.198478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bb7c6-f841-492a-ae8f-dd52bcba6d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:56.199183 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bb7c6-f841-492a-ae8f-dd52bcba6d6d None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:56.199472 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bb7c6-f841-492a-ae8f-dd52bcba6d6d None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:56.199752 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bb7c6-f841-492a-ae8f-dd52bcba6d6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:56.222042 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd233eb1-48ef-42a3-952b-45d75f61c93d None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:56.262772 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb004ca-025b-4fa0-8235-dc5f8d62d9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:56.263449 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb004ca-025b-4fa0-8235-dc5f8d62d9b6 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:56.263733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb004ca-025b-4fa0-8235-dc5f8d62d9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:56.264016 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb004ca-025b-4fa0-8235-dc5f8d62d9b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:56.264869 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb004ca-025b-4fa0-8235-dc5f8d62d9b6 None admin] RBAC: Adding query filter params (role.id=507fb7d64e4549c58ff859abf6160c43, scope.project.id=7be83fb6f20341139147373837320148, user.id=1c8f6262cf24419daf803ae71aed24f5) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:56.265182 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb004ca-025b-4fa0-8235-dc5f8d62d9b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:56.266654 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb004ca-025b-4fa0-8235-dc5f8d62d9b6 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:56.273808 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 565/1103] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:35:56 2022] GET /identity/v3/role_assignments?user.id=1c8f6262cf24419daf803ae71aed24f5&scope.project.id=7be83fb6f20341139147373837320148&role.id=507fb7d64e4549c58ff859abf6160c43 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:56.281004 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0bb7c6-f841-492a-ae8f-dd52bcba6d6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:56.282519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0bb7c6-f841-492a-ae8f-dd52bcba6d6d None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:56.356613 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:56.419698 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 539/1104] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:35:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:35:56.420874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb004ca-025b-4fa0-8235-dc5f8d62d9b6 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:56.452613 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:56.550504 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb004ca-025b-4fa0-8235-dc5f8d62d9b6 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:56.585325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e8d0d-ea3d-4c55-b63e-4309bd19e7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:56.586754 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e8d0d-ea3d-4c55-b63e-4309bd19e7c9 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:56.587312 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e8d0d-ea3d-4c55-b63e-4309bd19e7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:56.587312 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e8d0d-ea3d-4c55-b63e-4309bd19e7c9 None admin] PATH_INFO: `/v3/projects/6fddfead5e6c40e289cd1973ec85c380` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:56.588490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e8d0d-ea3d-4c55-b63e-4309bd19e7c9 None admin] RBAC: Authorizing `identity:get_project(project_id=6fddfead5e6c40e289cd1973ec85c380)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:56.589940 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e8d0d-ea3d-4c55-b63e-4309bd19e7c9 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:56.591730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 566/1105] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:35:56 2022] GET /identity/v3/projects/6fddfead5e6c40e289cd1973ec85c380 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:56.598116 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bb7c6-f841-492a-ae8f-dd52bcba6d6d None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:56.713913 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bb7c6-f841-492a-ae8f-dd52bcba6d6d None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:56.749444 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44334c0a-a786-4b4c-a338-725722e6705f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:56.750164 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44334c0a-a786-4b4c-a338-725722e6705f None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:56.750455 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44334c0a-a786-4b4c-a338-725722e6705f None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:56.750734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44334c0a-a786-4b4c-a338-725722e6705f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:56.751581 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44334c0a-a786-4b4c-a338-725722e6705f None admin] RBAC: Adding query filter params (role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.project.id=6fddfead5e6c40e289cd1973ec85c380, user.id=bb6dd937abbd432e946d06713440c7d1) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:35:56.751902 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44334c0a-a786-4b4c-a338-725722e6705f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:56.753351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44334c0a-a786-4b4c-a338-725722e6705f None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:56.760723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 540/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:35:56 2022] GET /identity/v3/role_assignments?user.id=bb6dd937abbd432e946d06713440c7d1&scope.project.id=6fddfead5e6c40e289cd1973ec85c380&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:35:58.881815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507baa8-6b64-4d69-8fa8-6c288f44a33f None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:58.882381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507baa8-6b64-4d69-8fa8-6c288f44a33f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:58.882381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507baa8-6b64-4d69-8fa8-6c288f44a33f None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:58.883755 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 567/1107] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:35:58.889670 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414ccc4-b61e-47eb-bcc0-a046ef45e49a None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:58.890880 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414ccc4-b61e-47eb-bcc0-a046ef45e49a None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:58.890880 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414ccc4-b61e-47eb-bcc0-a046ef45e49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:58.917948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-a414ccc4-b61e-47eb-bcc0-a046ef45e49a None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:35:58.919934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a414ccc4-b61e-47eb-bcc0-a046ef45e49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:35:58.946791 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:58.998523 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 541/1108] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:35:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:35:59.022946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507baa8-6b64-4d69-8fa8-6c288f44a33f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:59.111001 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:35:59.124339 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507baa8-6b64-4d69-8fa8-6c288f44a33f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:59.156721 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78311b-35fc-4882-bf2b-954522190092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:59.157492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78311b-35fc-4882-bf2b-954522190092 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:59.157808 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78311b-35fc-4882-bf2b-954522190092 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:59.158086 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78311b-35fc-4882-bf2b-954522190092 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:59.159236 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78311b-35fc-4882-bf2b-954522190092 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:59.160666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78311b-35fc-4882-bf2b-954522190092 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:59.162309 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 568/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:35:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:35:59.169084 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414ccc4-b61e-47eb-bcc0-a046ef45e49a None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:35:59.267490 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414ccc4-b61e-47eb-bcc0-a046ef45e49a None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:35:59.300157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fbbd6-4470-44e2-b109-0af20398f57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:35:59.300817 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fbbd6-4470-44e2-b109-0af20398f57e None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:35:59.301096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fbbd6-4470-44e2-b109-0af20398f57e None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:35:59.301367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fbbd6-4470-44e2-b109-0af20398f57e None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:35:59.302109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14fbbd6-4470-44e2-b109-0af20398f57e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:35:59.303538 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14fbbd6-4470-44e2-b109-0af20398f57e None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:35:59.324583 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 542/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jun 27 15:35:59 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:36:01.450950 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75119e98-0f36-406a-9128-8925edaf21ca None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:01.451519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75119e98-0f36-406a-9128-8925edaf21ca None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:01.451519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75119e98-0f36-406a-9128-8925edaf21ca None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:01.452873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 569/1111] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:36:01.458682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11497e-cdf3-4786-87f2-221776114d62 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:01.459101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11497e-cdf3-4786-87f2-221776114d62 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:01.459221 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11497e-cdf3-4786-87f2-221776114d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:01.476250 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-cb11497e-cdf3-4786-87f2-221776114d62 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:01.478133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-cb11497e-cdf3-4786-87f2-221776114d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:01.480177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:01.546718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 543/1112] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:36:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:36:01.570702 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75119e98-0f36-406a-9128-8925edaf21ca None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:01.685135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75119e98-0f36-406a-9128-8925edaf21ca None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:01.723761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d0235-b1d7-433c-8955-afc6ee6e32e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:01.724537 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d0235-b1d7-433c-8955-afc6ee6e32e6 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:01.724820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d0235-b1d7-433c-8955-afc6ee6e32e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:01.725095 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d0235-b1d7-433c-8955-afc6ee6e32e6 None admin] PATH_INFO: `/v3/roles/478b9cd589e846588025a4b4ed2c07f0` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:01.727680 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d0235-b1d7-433c-8955-afc6ee6e32e6 None admin] RBAC: Authorizing `identity:get_role(role_id=478b9cd589e846588025a4b4ed2c07f0)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:01.732376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d0235-b1d7-433c-8955-afc6ee6e32e6 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:01.733874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 570/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:36:01 2022] GET /identity/v3/roles/478b9cd589e846588025a4b4ed2c07f0 => generated 223 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:01.740134 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11497e-cdf3-4786-87f2-221776114d62 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:01.845731 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11497e-cdf3-4786-87f2-221776114d62 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:01.884036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f798bb-61d8-4936-aa99-54355ef052b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:01.884830 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f798bb-61d8-4936-aa99-54355ef052b9 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:01.885114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f798bb-61d8-4936-aa99-54355ef052b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:01.885392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f798bb-61d8-4936-aa99-54355ef052b9 None admin] PATH_INFO: `/v3/users/cdb12969e47e490ca8dae7ba53d5b07b` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:01.893075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:01.900906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f798bb-61d8-4936-aa99-54355ef052b9 None admin] RBAC: Authorizing `identity:get_user(user_id=cdb12969e47e490ca8dae7ba53d5b07b)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:01.902487 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f798bb-61d8-4936-aa99-54355ef052b9 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:01.904201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 544/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:36:01 2022] GET /identity/v3/users/cdb12969e47e490ca8dae7ba53d5b07b => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:01.909948 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d0235-b1d7-433c-8955-afc6ee6e32e6 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:01.935347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:02.020780 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d0235-b1d7-433c-8955-afc6ee6e32e6 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:02.055239 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a880f-fb58-42be-819b-dfa073e0938d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:02.055892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a880f-fb58-42be-819b-dfa073e0938d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:02.056177 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a880f-fb58-42be-819b-dfa073e0938d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:02.056449 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a880f-fb58-42be-819b-dfa073e0938d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:02.123094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a880f-fb58-42be-819b-dfa073e0938d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:02.124543 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a880f-fb58-42be-819b-dfa073e0938d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:02.147778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:02.224888 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 571/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:36:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:36:02.231514 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f798bb-61d8-4936-aa99-54355ef052b9 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:02.342782 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f798bb-61d8-4936-aa99-54355ef052b9 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:02.377829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35abbb-dade-401c-92c2-e680cd1140cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:02.378698 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35abbb-dade-401c-92c2-e680cd1140cf None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:02.378978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35abbb-dade-401c-92c2-e680cd1140cf None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:02.379257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35abbb-dade-401c-92c2-e680cd1140cf None admin] PATH_INFO: `/v3/projects/6fddfead5e6c40e289cd1973ec85c380` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:02.380455 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f35abbb-dade-401c-92c2-e680cd1140cf None admin] RBAC: Authorizing `identity:get_project(project_id=6fddfead5e6c40e289cd1973ec85c380)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:02.381962 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f35abbb-dade-401c-92c2-e680cd1140cf None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:02.383953 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 545/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:36:02 2022] GET /identity/v3/projects/6fddfead5e6c40e289cd1973ec85c380 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:02.390074 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a880f-fb58-42be-819b-dfa073e0938d None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:02.488807 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a880f-fb58-42be-819b-dfa073e0938d None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:02.513580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:02.524549 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e4728-6a6c-44e7-9fca-442068c9effa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:02.525201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e4728-6a6c-44e7-9fca-442068c9effa None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:02.525481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e4728-6a6c-44e7-9fca-442068c9effa None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:02.525783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e4728-6a6c-44e7-9fca-442068c9effa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:02.526610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e4728-6a6c-44e7-9fca-442068c9effa None admin] RBAC: Adding query filter params (role.id=478b9cd589e846588025a4b4ed2c07f0, scope.project.id=6fddfead5e6c40e289cd1973ec85c380, user.id=cdb12969e47e490ca8dae7ba53d5b07b) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:36:02.526920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e4728-6a6c-44e7-9fca-442068c9effa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:02.528340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e4728-6a6c-44e7-9fca-442068c9effa None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:02.533965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 572/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:36:02 2022] GET /identity/v3/role_assignments?user.id=cdb12969e47e490ca8dae7ba53d5b07b&scope.project.id=6fddfead5e6c40e289cd1973ec85c380&role.id=478b9cd589e846588025a4b4ed2c07f0 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:04.569394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed5479-0400-4344-9542-1988a76c8476 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:04.570001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed5479-0400-4344-9542-1988a76c8476 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:04.570001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed5479-0400-4344-9542-1988a76c8476 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:04.572750 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 546/1118] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:36:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:36:04.578427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aed602-92d8-467f-b1d1-207b75612a36 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:04.578900 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aed602-92d8-467f-b1d1-207b75612a36 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:04.578984 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aed602-92d8-467f-b1d1-207b75612a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:04.601023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-c8aed602-92d8-467f-b1d1-207b75612a36 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:04.602567 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-c8aed602-92d8-467f-b1d1-207b75612a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:04.669226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 573/1119] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:36:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:36:04.692709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed5479-0400-4344-9542-1988a76c8476 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:04.756876 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:04.797269 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eed5479-0400-4344-9542-1988a76c8476 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:04.831877 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e1e1d-6db7-493a-bb3c-2f05f00ece3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:04.832746 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e1e1d-6db7-493a-bb3c-2f05f00ece3b None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:04.833024 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e1e1d-6db7-493a-bb3c-2f05f00ece3b None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:04.833295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e1e1d-6db7-493a-bb3c-2f05f00ece3b None admin] PATH_INFO: `/v3/roles/478b9cd589e846588025a4b4ed2c07f0` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:04.834794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e1e1d-6db7-493a-bb3c-2f05f00ece3b None admin] RBAC: Authorizing `identity:get_role(role_id=478b9cd589e846588025a4b4ed2c07f0)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:04.836685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e1e1d-6db7-493a-bb3c-2f05f00ece3b None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:04.838111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 547/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:36:04 2022] GET /identity/v3/roles/478b9cd589e846588025a4b4ed2c07f0 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:04.844145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aed602-92d8-467f-b1d1-207b75612a36 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:04.941537 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aed602-92d8-467f-b1d1-207b75612a36 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:04.959533 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:04.976733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca8f8c-375a-4a5d-9496-26086941185b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:04.977605 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca8f8c-375a-4a5d-9496-26086941185b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:04.977919 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca8f8c-375a-4a5d-9496-26086941185b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:04.978224 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca8f8c-375a-4a5d-9496-26086941185b None admin] PATH_INFO: `/v3/users/cdb12969e47e490ca8dae7ba53d5b07b` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:04.980220 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca8f8c-375a-4a5d-9496-26086941185b None admin] RBAC: Authorizing `identity:get_user(user_id=cdb12969e47e490ca8dae7ba53d5b07b)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:04.982924 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca8f8c-375a-4a5d-9496-26086941185b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:04.985225 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 574/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:36:04 2022] GET /identity/v3/users/cdb12969e47e490ca8dae7ba53d5b07b => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:04.991506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e1e1d-6db7-493a-bb3c-2f05f00ece3b None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:05.090452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e1e1d-6db7-493a-bb3c-2f05f00ece3b None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:05.122415 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:05.127836 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50722e5-a071-4d79-acbd-7f7a99ccb1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:05.128710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50722e5-a071-4d79-acbd-7f7a99ccb1e3 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:05.128992 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50722e5-a071-4d79-acbd-7f7a99ccb1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:05.129270 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50722e5-a071-4d79-acbd-7f7a99ccb1e3 None admin] PATH_INFO: `/v3/projects/6fddfead5e6c40e289cd1973ec85c380` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:05.130465 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50722e5-a071-4d79-acbd-7f7a99ccb1e3 None admin] RBAC: Authorizing `identity:get_project(project_id=6fddfead5e6c40e289cd1973ec85c380)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:05.131912 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50722e5-a071-4d79-acbd-7f7a99ccb1e3 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:05.134260 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 548/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:36:04 2022] GET /identity/v3/projects/6fddfead5e6c40e289cd1973ec85c380 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:05.140164 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca8f8c-375a-4a5d-9496-26086941185b None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:05.241376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca8f8c-375a-4a5d-9496-26086941185b None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:05.275576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3c77b-f0f0-4b08-a324-bb323b89a2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:05.276453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3c77b-f0f0-4b08-a324-bb323b89a2cd None admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:05.276738 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3c77b-f0f0-4b08-a324-bb323b89a2cd None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:05.277015 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3c77b-f0f0-4b08-a324-bb323b89a2cd None admin] PATH_INFO: `/v3/projects/6fddfead5e6c40e289cd1973ec85c380/users/cdb12969e47e490ca8dae7ba53d5b07b/roles/478b9cd589e846588025a4b4ed2c07f0` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:05.279239 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3c77b-f0f0-4b08-a324-bb323b89a2cd None admin] RBAC: Authorizing `identity:create_grant(project_id=6fddfead5e6c40e289cd1973ec85c380, user_id=cdb12969e47e490ca8dae7ba53d5b07b, role_id=478b9cd589e846588025a4b4ed2c07f0)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:05.280658 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3c77b-f0f0-4b08-a324-bb323b89a2cd None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:05.294672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 575/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:36:05 2022] PUT /identity/v3/projects/6fddfead5e6c40e289cd1973ec85c380/users/cdb12969e47e490ca8dae7ba53d5b07b/roles/478b9cd589e846588025a4b4ed2c07f0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:36:07.332423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea1220-60a0-4c35-b4aa-11aadb2a419c None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:07.332954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea1220-60a0-4c35-b4aa-11aadb2a419c None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:07.332954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea1220-60a0-4c35-b4aa-11aadb2a419c None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:07.334386 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 549/1124] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:36:07.340606 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411da637-8fea-4c54-8747-629e5d95548f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:07.341026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411da637-8fea-4c54-8747-629e5d95548f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:07.341160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411da637-8fea-4c54-8747-629e5d95548f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:07.358178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-411da637-8fea-4c54-8747-629e5d95548f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:07.359355 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-411da637-8fea-4c54-8747-629e5d95548f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:07.400723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:07.426919 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 576/1125] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:36:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:36:07.451288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea1220-60a0-4c35-b4aa-11aadb2a419c None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:07.552632 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea1220-60a0-4c35-b4aa-11aadb2a419c None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:07.586512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53522a5c-b537-4fb7-a7a0-7c8eb97cda2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:07.587285 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53522a5c-b537-4fb7-a7a0-7c8eb97cda2a None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:07.587572 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53522a5c-b537-4fb7-a7a0-7c8eb97cda2a None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:07.587852 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53522a5c-b537-4fb7-a7a0-7c8eb97cda2a None admin] PATH_INFO: `/v3/roles/478b9cd589e846588025a4b4ed2c07f0` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:07.589335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53522a5c-b537-4fb7-a7a0-7c8eb97cda2a None admin] RBAC: Authorizing `identity:get_role(role_id=478b9cd589e846588025a4b4ed2c07f0)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:07.590748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53522a5c-b537-4fb7-a7a0-7c8eb97cda2a None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:07.592125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 550/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:36:07 2022] GET /identity/v3/roles/478b9cd589e846588025a4b4ed2c07f0 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:07.598074 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411da637-8fea-4c54-8747-629e5d95548f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:07.698145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411da637-8fea-4c54-8747-629e5d95548f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:07.731596 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff8463-019d-4b23-8322-6b719bd3fe81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:07.732464 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff8463-019d-4b23-8322-6b719bd3fe81 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:07.732747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff8463-019d-4b23-8322-6b719bd3fe81 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:07.733023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff8463-019d-4b23-8322-6b719bd3fe81 None admin] PATH_INFO: `/v3/users/cdb12969e47e490ca8dae7ba53d5b07b` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:07.734230 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff8463-019d-4b23-8322-6b719bd3fe81 None admin] RBAC: Authorizing `identity:get_user(user_id=cdb12969e47e490ca8dae7ba53d5b07b)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:07.735660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff8463-019d-4b23-8322-6b719bd3fe81 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:07.737285 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 577/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:36:07 2022] GET /identity/v3/users/cdb12969e47e490ca8dae7ba53d5b07b => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:07.743583 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53522a5c-b537-4fb7-a7a0-7c8eb97cda2a None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:07.809011 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:07.843295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53522a5c-b537-4fb7-a7a0-7c8eb97cda2a None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:07.876904 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd2660-6f17-4620-924f-81b9e4c384be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:07.877541 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd2660-6f17-4620-924f-81b9e4c384be None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:07.877846 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd2660-6f17-4620-924f-81b9e4c384be None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:07.878121 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd2660-6f17-4620-924f-81b9e4c384be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:07.943598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd2660-6f17-4620-924f-81b9e4c384be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:07.945053 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd2660-6f17-4620-924f-81b9e4c384be None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:08.008660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:08.045342 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 551/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:36:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:36:08.051216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff8463-019d-4b23-8322-6b719bd3fe81 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:08.097764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:08.154779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff8463-019d-4b23-8322-6b719bd3fe81 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:08.189388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06305265-b4a1-4246-9df1-9a1978b310a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:08.190275 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06305265-b4a1-4246-9df1-9a1978b310a8 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:08.190561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06305265-b4a1-4246-9df1-9a1978b310a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:08.190841 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06305265-b4a1-4246-9df1-9a1978b310a8 None admin] PATH_INFO: `/v3/projects/6fddfead5e6c40e289cd1973ec85c380` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:08.192010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06305265-b4a1-4246-9df1-9a1978b310a8 None admin] RBAC: Authorizing `identity:get_project(project_id=6fddfead5e6c40e289cd1973ec85c380)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:08.193437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06305265-b4a1-4246-9df1-9a1978b310a8 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:08.195777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 578/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:36:08 2022] GET /identity/v3/projects/6fddfead5e6c40e289cd1973ec85c380 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:08.201847 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd2660-6f17-4620-924f-81b9e4c384be None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:08.298876 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd2660-6f17-4620-924f-81b9e4c384be None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:08.332496 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeabfad-7caf-41e5-81af-2e40c1203bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:08.333144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeabfad-7caf-41e5-81af-2e40c1203bbe None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:08.333424 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeabfad-7caf-41e5-81af-2e40c1203bbe None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:08.333719 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeabfad-7caf-41e5-81af-2e40c1203bbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:08.334552 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeabfad-7caf-41e5-81af-2e40c1203bbe None admin] RBAC: Adding query filter params (role.id=478b9cd589e846588025a4b4ed2c07f0, scope.project.id=6fddfead5e6c40e289cd1973ec85c380, user.id=cdb12969e47e490ca8dae7ba53d5b07b) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:36:08.334858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeabfad-7caf-41e5-81af-2e40c1203bbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:08.336828 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeabfad-7caf-41e5-81af-2e40c1203bbe None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:08.342651 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 552/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:36:08 2022] GET /identity/v3/role_assignments?user.id=cdb12969e47e490ca8dae7ba53d5b07b&scope.project.id=6fddfead5e6c40e289cd1973ec85c380&role.id=478b9cd589e846588025a4b4ed2c07f0 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:10.496845 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4540d83-776f-478d-a278-80075b9a070d None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:10.497429 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4540d83-776f-478d-a278-80075b9a070d None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:10.497429 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4540d83-776f-478d-a278-80075b9a070d None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:10.498894 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 579/1131] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:36:10.505277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782110e-fdf1-4b55-bc99-46887acfb207 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:10.505899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782110e-fdf1-4b55-bc99-46887acfb207 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:10.505899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782110e-fdf1-4b55-bc99-46887acfb207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:10.524569 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-6782110e-fdf1-4b55-bc99-46887acfb207 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:10.525813 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-6782110e-fdf1-4b55-bc99-46887acfb207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:10.556050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:10.600906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 553/1132] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:36:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:36:10.628368 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4540d83-776f-478d-a278-80075b9a070d None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:10.733360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4540d83-776f-478d-a278-80075b9a070d None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:10.747879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:10.770830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc52774-e1c5-4488-a882-bea15792e40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:10.771617 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc52774-e1c5-4488-a882-bea15792e40b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:10.771898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc52774-e1c5-4488-a882-bea15792e40b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:10.772170 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc52774-e1c5-4488-a882-bea15792e40b None admin] PATH_INFO: `/v3/domains/default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:10.773400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc52774-e1c5-4488-a882-bea15792e40b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:10.774881 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc52774-e1c5-4488-a882-bea15792e40b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:10.776535 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 580/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:36:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:36:10.783069 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782110e-fdf1-4b55-bc99-46887acfb207 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:10.894345 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782110e-fdf1-4b55-bc99-46887acfb207 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:10.934326 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565631d4-1c66-47c9-8a38-82cb8b393ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:10.935013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565631d4-1c66-47c9-8a38-82cb8b393ffd None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:10.935300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565631d4-1c66-47c9-8a38-82cb8b393ffd None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:10.935583 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565631d4-1c66-47c9-8a38-82cb8b393ffd None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:10.936309 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565631d4-1c66-47c9-8a38-82cb8b393ffd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:10.937801 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565631d4-1c66-47c9-8a38-82cb8b393ffd None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:10.967636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 554/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jun 27 15:36:10 2022] POST /identity/v3/projects => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:36:13.142953 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed914ac0-c4dd-45f1-ace3-a04cb5195ab2 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:13.143585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed914ac0-c4dd-45f1-ace3-a04cb5195ab2 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:13.143585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed914ac0-c4dd-45f1-ace3-a04cb5195ab2 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:13.145069 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 581/1135] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:36:13.152376 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e25ff-f280-4e1b-be49-e378d401a66e None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:13.152934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e25ff-f280-4e1b-be49-e378d401a66e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:13.153101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e25ff-f280-4e1b-be49-e378d401a66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:13.172087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-275e25ff-f280-4e1b-be49-e378d401a66e None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:13.173420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-275e25ff-f280-4e1b-be49-e378d401a66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:13.189095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:13.268617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 555/1136] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:36:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:36:13.296321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed914ac0-c4dd-45f1-ace3-a04cb5195ab2 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:13.418444 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed914ac0-c4dd-45f1-ace3-a04cb5195ab2 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:13.459340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21dc5f1-c2e2-473c-a634-b4c0ac998d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:13.460148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dc5f1-c2e2-473c-a634-b4c0ac998d5e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:13.460438 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dc5f1-c2e2-473c-a634-b4c0ac998d5e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:13.461439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dc5f1-c2e2-473c-a634-b4c0ac998d5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:13.462818 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21dc5f1-c2e2-473c-a634-b4c0ac998d5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:13.464334 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21dc5f1-c2e2-473c-a634-b4c0ac998d5e None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:13.466164 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 582/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:36:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:13.475091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e25ff-f280-4e1b-be49-e378d401a66e None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:13.594735 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e25ff-f280-4e1b-be49-e378d401a66e None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:13.635875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b9002-a66d-4976-b1b4-b3576773712e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:13.636622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b9002-a66d-4976-b1b4-b3576773712e None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:13.636969 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b9002-a66d-4976-b1b4-b3576773712e None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:13.637294 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b9002-a66d-4976-b1b4-b3576773712e None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:13.638128 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b9002-a66d-4976-b1b4-b3576773712e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:13.639759 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b9002-a66d-4976-b1b4-b3576773712e None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:13.658065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:13.664911 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 556/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jun 27 15:36:13 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:36:15.856820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d741a-b77f-4b07-a8ca-7e3a15fdb850 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:15.857377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d741a-b77f-4b07-a8ca-7e3a15fdb850 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:15.857377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d741a-b77f-4b07-a8ca-7e3a15fdb850 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:15.858940 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 583/1139] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:36:15.864854 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdac84-11e3-4640-8554-3cc964f3d529 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:15.865419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdac84-11e3-4640-8554-3cc964f3d529 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:15.865419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdac84-11e3-4640-8554-3cc964f3d529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:15.884617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-4afdac84-11e3-4640-8554-3cc964f3d529 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:15.885891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4afdac84-11e3-4640-8554-3cc964f3d529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:15.958342 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 557/1140] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:36:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:36:15.985669 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d741a-b77f-4b07-a8ca-7e3a15fdb850 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:16.043437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:16.109589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d741a-b77f-4b07-a8ca-7e3a15fdb850 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:16.146612 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16331ede-1cec-48f3-8062-c7c8fa96c11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:16.147893 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16331ede-1cec-48f3-8062-c7c8fa96c11d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:16.148924 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16331ede-1cec-48f3-8062-c7c8fa96c11d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:16.149450 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16331ede-1cec-48f3-8062-c7c8fa96c11d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:16.168778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16331ede-1cec-48f3-8062-c7c8fa96c11d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:36:16.183641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-16331ede-1cec-48f3-8062-c7c8fa96c11d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:36:16.183641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:36:16.183641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:36:16.183641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:36:16.183641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:36:16.183641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:36:16.183641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:36:16.183641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:36:16.183641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:36:16.183641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:36:16.183641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:36:16.183641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:36:16.183641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:36:16.183641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:36:16.183641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:36:16.183641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:36:16.185325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:36:16.185325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:36:16.185325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:36:16.185325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:36:16.185325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:36:16.185325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:36:16.185325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:36:16.185325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:36:16.185325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:36:16.185325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:36:16.185325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:36:16.185325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:36:16.185325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:36:16.185325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:36:16.185325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:36:16.185325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:36:16.186986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:36:16.186986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:36:16.186986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:36:16.186986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:36:16.186986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:36:16.186986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:36:16.186986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:36:16.186986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:36:16.186986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:36:16.186986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:36:16.186986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:36:16.186986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:36:16.186986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 584/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:36:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:36:16.191773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afdac84-11e3-4640-8554-3cc964f3d529 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:16.306817 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afdac84-11e3-4640-8554-3cc964f3d529 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:16.341307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:16.349742 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc59a6e-f7ff-4a12-bb5e-64e435c0cc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:16.350727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc59a6e-f7ff-4a12-bb5e-64e435c0cc1f None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:16.351150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc59a6e-f7ff-4a12-bb5e-64e435c0cc1f None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:16.351523 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc59a6e-f7ff-4a12-bb5e-64e435c0cc1f None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:16.352423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc59a6e-f7ff-4a12-bb5e-64e435c0cc1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:36:16.352831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc59a6e-f7ff-4a12-bb5e-64e435c0cc1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:16.354871 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc59a6e-f7ff-4a12-bb5e-64e435c0cc1f None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:16.368434 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 558/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:36:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:16.377828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16331ede-1cec-48f3-8062-c7c8fa96c11d None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:16.488382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16331ede-1cec-48f3-8062-c7c8fa96c11d None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:16.498418 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:16.526889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5c8be-499c-47ad-8f7c-5bbefb3c5ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:16.527780 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5c8be-499c-47ad-8f7c-5bbefb3c5ac2 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:16.528060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5c8be-499c-47ad-8f7c-5bbefb3c5ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:16.528333 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5c8be-499c-47ad-8f7c-5bbefb3c5ac2 None admin] PATH_INFO: `/v3/users/7e0fe8f0e2e04d838e7293926b4ad49d` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:16.544329 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5c8be-499c-47ad-8f7c-5bbefb3c5ac2 None admin] RBAC: Authorizing `identity:get_user(user_id=7e0fe8f0e2e04d838e7293926b4ad49d)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:16.545820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5c8be-499c-47ad-8f7c-5bbefb3c5ac2 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:16.547597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 585/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:36:16 2022] GET /identity/v3/users/7e0fe8f0e2e04d838e7293926b4ad49d => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:16.553653 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc59a6e-f7ff-4a12-bb5e-64e435c0cc1f None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:16.692734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc59a6e-f7ff-4a12-bb5e-64e435c0cc1f None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:16.766433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058e8fa-8bcb-4e09-9eda-0654d8a35fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:16.767518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058e8fa-8bcb-4e09-9eda-0654d8a35fc6 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:16.767997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058e8fa-8bcb-4e09-9eda-0654d8a35fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:16.768475 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058e8fa-8bcb-4e09-9eda-0654d8a35fc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:16.879548 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:16.896379 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058e8fa-8bcb-4e09-9eda-0654d8a35fc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:16.898986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058e8fa-8bcb-4e09-9eda-0654d8a35fc6 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:17.078436 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 559/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:36:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 15:36:17.084422 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5c8be-499c-47ad-8f7c-5bbefb3c5ac2 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:17.200050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5c8be-499c-47ad-8f7c-5bbefb3c5ac2 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:17.240612 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b383dd-2249-4aef-bd23-2af9117e3f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:17.241481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b383dd-2249-4aef-bd23-2af9117e3f80 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:17.242559 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b383dd-2249-4aef-bd23-2af9117e3f80 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:17.244541 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b383dd-2249-4aef-bd23-2af9117e3f80 None admin] PATH_INFO: `/v3/projects/838eaca6365c43f6a8518c1f1b246493` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:17.246460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b383dd-2249-4aef-bd23-2af9117e3f80 None admin] RBAC: Authorizing `identity:get_project(project_id=838eaca6365c43f6a8518c1f1b246493)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:17.248175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b383dd-2249-4aef-bd23-2af9117e3f80 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:17.250288 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 586/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:36:17 2022] GET /identity/v3/projects/838eaca6365c43f6a8518c1f1b246493 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:36:17.260821 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058e8fa-8bcb-4e09-9eda-0654d8a35fc6 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:17.368492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:17.378646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058e8fa-8bcb-4e09-9eda-0654d8a35fc6 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:17.421392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b81b4-8a74-44c3-b1eb-d9ba4c884d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:17.422116 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b81b4-8a74-44c3-b1eb-d9ba4c884d3f None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:17.422406 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b81b4-8a74-44c3-b1eb-d9ba4c884d3f None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:17.422689 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b81b4-8a74-44c3-b1eb-d9ba4c884d3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:17.423578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b81b4-8a74-44c3-b1eb-d9ba4c884d3f None admin] RBAC: Adding query filter params (role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.project.id=838eaca6365c43f6a8518c1f1b246493, user.id=7e0fe8f0e2e04d838e7293926b4ad49d) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:36:17.423900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b81b4-8a74-44c3-b1eb-d9ba4c884d3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:17.425389 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b81b4-8a74-44c3-b1eb-d9ba4c884d3f None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:17.432087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 560/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:36:17 2022] GET /identity/v3/role_assignments?user.id=7e0fe8f0e2e04d838e7293926b4ad49d&scope.project.id=838eaca6365c43f6a8518c1f1b246493&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:19.545310 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102801f-7415-46dc-b609-203b734441a6 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:19.545929 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102801f-7415-46dc-b609-203b734441a6 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:19.545929 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102801f-7415-46dc-b609-203b734441a6 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:19.547490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 587/1147] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:36:19.553279 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca605a-8882-4010-8132-c55172ab25fd None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:19.553826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca605a-8882-4010-8132-c55172ab25fd None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:19.553972 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca605a-8882-4010-8132-c55172ab25fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:19.571651 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-cbca605a-8882-4010-8132-c55172ab25fd None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:19.572859 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-cbca605a-8882-4010-8132-c55172ab25fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:19.647650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 561/1148] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:36:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:36:19.672954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102801f-7415-46dc-b609-203b734441a6 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:19.712638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:19.787606 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102801f-7415-46dc-b609-203b734441a6 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:19.827590 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c36abe-0f6d-4af9-b0b5-25fd314ba4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:19.828357 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c36abe-0f6d-4af9-b0b5-25fd314ba4bc None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:19.828574 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c36abe-0f6d-4af9-b0b5-25fd314ba4bc None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:19.828851 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c36abe-0f6d-4af9-b0b5-25fd314ba4bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:19.840291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c36abe-0f6d-4af9-b0b5-25fd314ba4bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:36:19.848110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-86c36abe-0f6d-4af9-b0b5-25fd314ba4bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:36:19.848110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:36:19.848110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:36:19.848110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:36:19.848110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:36:19.848110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:36:19.848110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:36:19.848110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:36:19.848110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:36:19.848110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:36:19.848110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:36:19.848110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:36:19.848110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:36:19.848110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:36:19.848110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:36:19.848110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:36:19.849938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:36:19.849938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:36:19.849938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:36:19.849938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:36:19.849938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:36:19.849938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:36:19.849938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:36:19.849938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:36:19.849938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:36:19.849938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:36:19.849938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:36:19.849938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:36:19.849938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:36:19.849938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:36:19.849938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:36:19.849938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:36:19.852038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:36:19.852038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:36:19.852038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:36:19.852038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:36:19.852038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:36:19.852038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:36:19.852038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:36:19.852038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:36:19.852038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:36:19.852038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:36:19.852038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:36:19.852038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:36:19.852038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 588/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:36:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:36:19.856896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca605a-8882-4010-8132-c55172ab25fd None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:19.997472 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:20.022241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca605a-8882-4010-8132-c55172ab25fd None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:20.081053 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e63e33-d7c7-43d2-b582-4fa3430088c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:20.082104 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e63e33-d7c7-43d2-b582-4fa3430088c2 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:20.082537 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e63e33-d7c7-43d2-b582-4fa3430088c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:20.082960 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e63e33-d7c7-43d2-b582-4fa3430088c2 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:20.084152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e63e33-d7c7-43d2-b582-4fa3430088c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:36:20.084646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e63e33-d7c7-43d2-b582-4fa3430088c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:20.087178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e63e33-d7c7-43d2-b582-4fa3430088c2 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:20.103123 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 562/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:36:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:20.109208 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c36abe-0f6d-4af9-b0b5-25fd314ba4bc None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:20.212462 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:20.220439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c36abe-0f6d-4af9-b0b5-25fd314ba4bc None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:20.259746 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102d6a5-e08a-44b9-a8e5-c04e9f7db742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:20.260685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102d6a5-e08a-44b9-a8e5-c04e9f7db742 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:20.261014 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102d6a5-e08a-44b9-a8e5-c04e9f7db742 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:20.261331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102d6a5-e08a-44b9-a8e5-c04e9f7db742 None admin] PATH_INFO: `/v3/users/7e0fe8f0e2e04d838e7293926b4ad49d` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:20.262715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102d6a5-e08a-44b9-a8e5-c04e9f7db742 None admin] RBAC: Authorizing `identity:get_user(user_id=7e0fe8f0e2e04d838e7293926b4ad49d)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:20.264236 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102d6a5-e08a-44b9-a8e5-c04e9f7db742 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:20.266059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 589/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:36:20 2022] GET /identity/v3/users/7e0fe8f0e2e04d838e7293926b4ad49d => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:36:20.272214 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e63e33-d7c7-43d2-b582-4fa3430088c2 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:20.386957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e63e33-d7c7-43d2-b582-4fa3430088c2 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:20.422105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01ec9f-5857-4adc-8dbc-67396ec06ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:20.422971 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01ec9f-5857-4adc-8dbc-67396ec06ae6 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:20.423259 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01ec9f-5857-4adc-8dbc-67396ec06ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:20.423537 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01ec9f-5857-4adc-8dbc-67396ec06ae6 None admin] PATH_INFO: `/v3/projects/838eaca6365c43f6a8518c1f1b246493` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:20.424728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01ec9f-5857-4adc-8dbc-67396ec06ae6 None admin] RBAC: Authorizing `identity:get_project(project_id=838eaca6365c43f6a8518c1f1b246493)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:20.426223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01ec9f-5857-4adc-8dbc-67396ec06ae6 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:20.427947 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 563/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:36:20 2022] GET /identity/v3/projects/838eaca6365c43f6a8518c1f1b246493 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:20.434575 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102d6a5-e08a-44b9-a8e5-c04e9f7db742 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:20.545543 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102d6a5-e08a-44b9-a8e5-c04e9f7db742 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:20.580662 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f1283-0142-43cb-996d-42b6eaea0f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:20.581566 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f1283-0142-43cb-996d-42b6eaea0f71 None admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:20.581880 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f1283-0142-43cb-996d-42b6eaea0f71 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:20.582156 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f1283-0142-43cb-996d-42b6eaea0f71 None admin] PATH_INFO: `/v3/projects/838eaca6365c43f6a8518c1f1b246493/users/7e0fe8f0e2e04d838e7293926b4ad49d/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:20.584141 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f1283-0142-43cb-996d-42b6eaea0f71 None admin] RBAC: Authorizing `identity:create_grant(project_id=838eaca6365c43f6a8518c1f1b246493, user_id=7e0fe8f0e2e04d838e7293926b4ad49d, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:20.586141 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f1283-0142-43cb-996d-42b6eaea0f71 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:20.601949 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 590/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:36:20 2022] PUT /identity/v3/projects/838eaca6365c43f6a8518c1f1b246493/users/7e0fe8f0e2e04d838e7293926b4ad49d/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:36:22.733112 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca009f4-eaa5-48db-bf82-19f08579e50a None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:22.733782 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca009f4-eaa5-48db-bf82-19f08579e50a None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:22.733782 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca009f4-eaa5-48db-bf82-19f08579e50a None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:22.735101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 564/1154] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:36:22.740639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9063f9a-52b9-4148-b93a-274897dab98d None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:22.741081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9063f9a-52b9-4148-b93a-274897dab98d None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:22.741287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9063f9a-52b9-4148-b93a-274897dab98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:22.760381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-f9063f9a-52b9-4148-b93a-274897dab98d None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:22.761594 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f9063f9a-52b9-4148-b93a-274897dab98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:22.779700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:22.838244 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 591/1155] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:36:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:36:22.862476 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca009f4-eaa5-48db-bf82-19f08579e50a None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:22.882890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:22.974049 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca009f4-eaa5-48db-bf82-19f08579e50a None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:23.011171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d423c2-5157-4b7a-a850-32f857c04475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:23.011858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d423c2-5157-4b7a-a850-32f857c04475 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:23.012143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d423c2-5157-4b7a-a850-32f857c04475 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:23.012419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d423c2-5157-4b7a-a850-32f857c04475 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:23.023927 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d423c2-5157-4b7a-a850-32f857c04475 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:36:23.031646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-b4d423c2-5157-4b7a-a850-32f857c04475 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:36:23.031646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:36:23.031646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:36:23.031646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:36:23.031646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:36:23.031646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:36:23.031646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:36:23.031646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:36:23.031646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:36:23.031646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:36:23.031646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:36:23.031646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:36:23.031646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:36:23.031646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:36:23.031646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:36:23.031646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:36:23.033394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:36:23.033394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:36:23.033394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:36:23.033394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:36:23.033394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:36:23.033394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:36:23.033394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:36:23.033394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:36:23.033394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:36:23.033394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:36:23.033394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:36:23.033394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:36:23.033394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:36:23.033394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:36:23.033394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:36:23.033394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:36:23.034612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:36:23.034612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:36:23.034612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:36:23.034612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:36:23.034612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:36:23.034612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:36:23.034612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:36:23.034612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:36:23.034612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:36:23.034612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:36:23.034612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:36:23.034612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:36:23.034612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 565/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:36:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:36:23.042745 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9063f9a-52b9-4148-b93a-274897dab98d None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:23.151627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9063f9a-52b9-4148-b93a-274897dab98d None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:23.185284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5135a9-fba9-4c00-a721-33f9c4fe8b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:23.186076 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5135a9-fba9-4c00-a721-33f9c4fe8b4a None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:23.186362 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5135a9-fba9-4c00-a721-33f9c4fe8b4a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:23.186635 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5135a9-fba9-4c00-a721-33f9c4fe8b4a None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:23.187387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5135a9-fba9-4c00-a721-33f9c4fe8b4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:36:23.187692 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5135a9-fba9-4c00-a721-33f9c4fe8b4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:23.189062 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5135a9-fba9-4c00-a721-33f9c4fe8b4a None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:23.198446 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 592/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:36:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:36:23.205405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d423c2-5157-4b7a-a850-32f857c04475 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:23.283389 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:23.308347 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d423c2-5157-4b7a-a850-32f857c04475 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:23.347507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fa9de-564b-4829-829b-27bdbb49734a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:23.348929 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fa9de-564b-4829-829b-27bdbb49734a None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:23.349250 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fa9de-564b-4829-829b-27bdbb49734a None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:23.349591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fa9de-564b-4829-829b-27bdbb49734a None admin] PATH_INFO: `/v3/users/7e0fe8f0e2e04d838e7293926b4ad49d` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:23.351505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fa9de-564b-4829-829b-27bdbb49734a None admin] RBAC: Authorizing `identity:get_user(user_id=7e0fe8f0e2e04d838e7293926b4ad49d)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:23.352998 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fa9de-564b-4829-829b-27bdbb49734a None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:23.354784 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 566/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:36:23 2022] GET /identity/v3/users/7e0fe8f0e2e04d838e7293926b4ad49d => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:36:23.364192 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5135a9-fba9-4c00-a721-33f9c4fe8b4a None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:23.391476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:23.528791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5135a9-fba9-4c00-a721-33f9c4fe8b4a None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:23.582380 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddd15b-5658-4d0b-94b8-0509d39031f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:23.583402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddd15b-5658-4d0b-94b8-0509d39031f5 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:23.583853 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddd15b-5658-4d0b-94b8-0509d39031f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:23.584314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddd15b-5658-4d0b-94b8-0509d39031f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:23.694296 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ddd15b-5658-4d0b-94b8-0509d39031f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:23.697094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ddd15b-5658-4d0b-94b8-0509d39031f5 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:23.721031 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:23.837889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 593/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:36:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:36:23.843815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fa9de-564b-4829-829b-27bdbb49734a None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:23.943618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fa9de-564b-4829-829b-27bdbb49734a None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:23.984344 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05e8c8-bc67-407d-b552-88b19822706e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:23.985815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05e8c8-bc67-407d-b552-88b19822706e None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:23.986301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05e8c8-bc67-407d-b552-88b19822706e None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:23.986798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05e8c8-bc67-407d-b552-88b19822706e None admin] PATH_INFO: `/v3/projects/838eaca6365c43f6a8518c1f1b246493` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:23.988792 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05e8c8-bc67-407d-b552-88b19822706e None admin] RBAC: Authorizing `identity:get_project(project_id=838eaca6365c43f6a8518c1f1b246493)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:23.991469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05e8c8-bc67-407d-b552-88b19822706e None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:23.995112 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 567/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:36:23 2022] GET /identity/v3/projects/838eaca6365c43f6a8518c1f1b246493 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:24.001213 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddd15b-5658-4d0b-94b8-0509d39031f5 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:24.103478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddd15b-5658-4d0b-94b8-0509d39031f5 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:24.118573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:24.138984 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6335b6-73d0-42dd-a21d-64278d892ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:24.139643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6335b6-73d0-42dd-a21d-64278d892ccc None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:24.139929 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6335b6-73d0-42dd-a21d-64278d892ccc None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:24.140208 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6335b6-73d0-42dd-a21d-64278d892ccc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:24.141047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6335b6-73d0-42dd-a21d-64278d892ccc None admin] RBAC: Adding query filter params (role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.project.id=838eaca6365c43f6a8518c1f1b246493, user.id=7e0fe8f0e2e04d838e7293926b4ad49d) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:36:24.141362 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6335b6-73d0-42dd-a21d-64278d892ccc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:24.142835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6335b6-73d0-42dd-a21d-64278d892ccc None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:24.149630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 594/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:36:24 2022] GET /identity/v3/role_assignments?user.id=7e0fe8f0e2e04d838e7293926b4ad49d&scope.project.id=838eaca6365c43f6a8518c1f1b246493&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:26.262902 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc0de6-84f3-4065-bc04-bbf8cd18feca None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:26.263494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc0de6-84f3-4065-bc04-bbf8cd18feca None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:26.263494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc0de6-84f3-4065-bc04-bbf8cd18feca None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:26.264825 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 568/1162] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:36:26.270432 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cba7c-6ae3-4291-a2d2-6d5addd766fe None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:26.270862 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cba7c-6ae3-4291-a2d2-6d5addd766fe None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:26.270985 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5cba7c-6ae3-4291-a2d2-6d5addd766fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:26.287969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-ac5cba7c-6ae3-4291-a2d2-6d5addd766fe None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:26.290062 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-ac5cba7c-6ae3-4291-a2d2-6d5addd766fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:26.358688 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 595/1163] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:36:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:36:26.384004 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc0de6-84f3-4065-bc04-bbf8cd18feca None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:26.401141 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:26.508875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc0de6-84f3-4065-bc04-bbf8cd18feca None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:26.546380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef6140-4023-4a08-b691-213055e49a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:26.546950 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef6140-4023-4a08-b691-213055e49a60 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:26.547235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef6140-4023-4a08-b691-213055e49a60 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:26.547509 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef6140-4023-4a08-b691-213055e49a60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:26.622927 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef6140-4023-4a08-b691-213055e49a60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:26.624412 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef6140-4023-4a08-b691-213055e49a60 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:26.639527 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:26.734458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 569/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:36:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:36:26.740772 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cba7c-6ae3-4291-a2d2-6d5addd766fe None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:26.847704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5cba7c-6ae3-4291-a2d2-6d5addd766fe None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:26.857754 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:26.884517 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25538f-4245-4708-82b4-9507fa22af0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:26.885307 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25538f-4245-4708-82b4-9507fa22af0e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:26.885588 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25538f-4245-4708-82b4-9507fa22af0e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:26.885896 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25538f-4245-4708-82b4-9507fa22af0e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:26.892920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25538f-4245-4708-82b4-9507fa22af0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:26.894413 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25538f-4245-4708-82b4-9507fa22af0e None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:26.908946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-6c25538f-4245-4708-82b4-9507fa22af0e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 27 15:36:26.908946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:36:26.908946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 27 15:36:26.908946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 27 15:36:26.908946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:36:26.908946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:36:26.908946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:36:26.908946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:36:26.908946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 27 15:36:26.908946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:36:26.908946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:36:26.908946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:36:26.908946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:36:26.908946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:36:26.908946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:36:26.908946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:36:26.908946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:36:26.910278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:36:26.910278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:36:26.910278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:36:26.910278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:36:26.910278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:36:26.910278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:36:26.910278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:36:26.910278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:36:26.910278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:36:26.910278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:36:26.910278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:36:26.910278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:36:26.910278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:36:26.910278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:36:26.910278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:36:26.910278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:36:26.911178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:36:26.911178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:36:26.911178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:36:26.911178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:36:26.911178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:36:26.911178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:36:26.911178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:36:26.911178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:36:26.911178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:36:26.911178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:36:26.911178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 27 15:36:26.911178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:36:26.911178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 27 15:36:26.911178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:36:26.911178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 596/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jun 27 15:36:26 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:36:26.917331 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef6140-4023-4a08-b691-213055e49a60 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:27.025787 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef6140-4023-4a08-b691-213055e49a60 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:27.040485 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:27.062521 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9327e26-1f6e-4c33-87cf-619cba3c9748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:27.063192 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9327e26-1f6e-4c33-87cf-619cba3c9748 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:27.063479 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9327e26-1f6e-4c33-87cf-619cba3c9748 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:27.063766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9327e26-1f6e-4c33-87cf-619cba3c9748 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:27.064495 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9327e26-1f6e-4c33-87cf-619cba3c9748 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:36:27.064806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9327e26-1f6e-4c33-87cf-619cba3c9748 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:27.066306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9327e26-1f6e-4c33-87cf-619cba3c9748 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:27.074278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 570/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Mon Jun 27 15:36:26 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:36:29.157917 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2aac1-5c5b-49e2-a238-942ee6ad21a7 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:29.158478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2aac1-5c5b-49e2-a238-942ee6ad21a7 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:29.158478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2aac1-5c5b-49e2-a238-942ee6ad21a7 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:29.159854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 597/1167] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:36:29.166352 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c697df-7f3d-468d-bbd6-c312eee72cca None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:29.166837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c697df-7f3d-468d-bbd6-c312eee72cca None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:29.166937 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c697df-7f3d-468d-bbd6-c312eee72cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:29.184448 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-89c697df-7f3d-468d-bbd6-c312eee72cca None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:29.185668 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-89c697df-7f3d-468d-bbd6-c312eee72cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:29.253834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 571/1168] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:36:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:36:29.277861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2aac1-5c5b-49e2-a238-942ee6ad21a7 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:29.378771 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2aac1-5c5b-49e2-a238-942ee6ad21a7 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:29.413818 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3ed56-a7ec-43ae-af2e-7fe51bd21483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:29.414398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3ed56-a7ec-43ae-af2e-7fe51bd21483 None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:29.414683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3ed56-a7ec-43ae-af2e-7fe51bd21483 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:29.415216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3ed56-a7ec-43ae-af2e-7fe51bd21483 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:29.415964 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3ed56-a7ec-43ae-af2e-7fe51bd21483 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:29.417320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3ed56-a7ec-43ae-af2e-7fe51bd21483 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:29.432368 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 598/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Mon Jun 27 15:36:29 2022] POST /identity/v3/domains => generated 267 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:36:31.562672 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7ec4f-97db-4f22-9cc4-6dc52fca615c None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:31.563267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7ec4f-97db-4f22-9cc4-6dc52fca615c None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:31.563267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7ec4f-97db-4f22-9cc4-6dc52fca615c None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:31.564657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 572/1170] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:36:31.570989 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b55a61-262e-47c7-9648-994ace6d01b9 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:31.571518 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b55a61-262e-47c7-9648-994ace6d01b9 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:31.571615 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b55a61-262e-47c7-9648-994ace6d01b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:31.590123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-05b55a61-262e-47c7-9648-994ace6d01b9 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:31.591337 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-05b55a61-262e-47c7-9648-994ace6d01b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:31.621293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:31.672019 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 599/1171] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:36:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:36:31.720795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7ec4f-97db-4f22-9cc4-6dc52fca615c None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:31.827837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7ec4f-97db-4f22-9cc4-6dc52fca615c None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:31.836504 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:31.864950 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4055ac-0972-4c87-b83a-86f0cf8a7177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:31.865861 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4055ac-0972-4c87-b83a-86f0cf8a7177 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:31.866149 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4055ac-0972-4c87-b83a-86f0cf8a7177 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:31.866637 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4055ac-0972-4c87-b83a-86f0cf8a7177 None admin] PATH_INFO: `/v3/domains/a460f94ecc314c2396a97ce7b76db9f0` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:31.878362 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4055ac-0972-4c87-b83a-86f0cf8a7177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a460f94ecc314c2396a97ce7b76db9f0)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:31.879949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4055ac-0972-4c87-b83a-86f0cf8a7177 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:31.881696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 573/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Mon Jun 27 15:36:31 2022] GET /identity/v3/domains/a460f94ecc314c2396a97ce7b76db9f0 => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:31.888533 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b55a61-262e-47c7-9648-994ace6d01b9 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:32.013967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b55a61-262e-47c7-9648-994ace6d01b9 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:32.050668 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a1607-b05e-4a5a-9863-a6b3a858c143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:32.051330 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a1607-b05e-4a5a-9863-a6b3a858c143 None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:32.051615 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a1607-b05e-4a5a-9863-a6b3a858c143 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:32.051894 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a1607-b05e-4a5a-9863-a6b3a858c143 None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:32.052607 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a1607-b05e-4a5a-9863-a6b3a858c143 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:32.054331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a1607-b05e-4a5a-9863-a6b3a858c143 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:32.081358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 600/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jun 27 15:36:31 2022] POST /identity/v3/projects => generated 361 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:36:34.234682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacd847-8ae6-487a-9fd6-1ccde5841c3e None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:34.235243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacd847-8ae6-487a-9fd6-1ccde5841c3e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:34.235243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacd847-8ae6-487a-9fd6-1ccde5841c3e None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:34.236666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 574/1174] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:36:34.243379 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6152ab-2e63-4e2a-a889-a3e48be292fe None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:34.243937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6152ab-2e63-4e2a-a889-a3e48be292fe None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:34.244369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6152ab-2e63-4e2a-a889-a3e48be292fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:34.264773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-ee6152ab-2e63-4e2a-a889-a3e48be292fe None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:34.266041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-ee6152ab-2e63-4e2a-a889-a3e48be292fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:34.269037 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:34.339934 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 601/1175] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:36:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:36:34.369091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eacd847-8ae6-487a-9fd6-1ccde5841c3e None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:34.480407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eacd847-8ae6-487a-9fd6-1ccde5841c3e None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:34.517314 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e583dd-9fe3-4013-896b-5ccf4a54b292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:34.518153 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e583dd-9fe3-4013-896b-5ccf4a54b292 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:34.518441 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e583dd-9fe3-4013-896b-5ccf4a54b292 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:34.518723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e583dd-9fe3-4013-896b-5ccf4a54b292 None admin] PATH_INFO: `/v3/domains/a460f94ecc314c2396a97ce7b76db9f0` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:34.520005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e583dd-9fe3-4013-896b-5ccf4a54b292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a460f94ecc314c2396a97ce7b76db9f0)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:34.521490 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e583dd-9fe3-4013-896b-5ccf4a54b292 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:34.524136 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 575/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Mon Jun 27 15:36:34 2022] GET /identity/v3/domains/a460f94ecc314c2396a97ce7b76db9f0 => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:34.530181 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6152ab-2e63-4e2a-a889-a3e48be292fe None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:34.634663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6152ab-2e63-4e2a-a889-a3e48be292fe None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:34.670705 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a39e39-8c3f-4ef1-adc1-75962bacdc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:34.671378 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a39e39-8c3f-4ef1-adc1-75962bacdc36 None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:34.671663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a39e39-8c3f-4ef1-adc1-75962bacdc36 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:34.671939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a39e39-8c3f-4ef1-adc1-75962bacdc36 None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:34.672663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a39e39-8c3f-4ef1-adc1-75962bacdc36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:34.674156 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a39e39-8c3f-4ef1-adc1-75962bacdc36 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:34.684701 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:34.698314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 602/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jun 27 15:36:34 2022] POST /identity/v3/users => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:36:36.881430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fff2e-e924-4e6c-bbca-d77b367cf851 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:36.882808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fff2e-e924-4e6c-bbca-d77b367cf851 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:36.882808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fff2e-e924-4e6c-bbca-d77b367cf851 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:36.884580 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 576/1178] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:36:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:36:36.890156 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497b85d-d83b-4dab-8e8c-3e9b3488b72d None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:36.890597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497b85d-d83b-4dab-8e8c-3e9b3488b72d None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:36.890696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5497b85d-d83b-4dab-8e8c-3e9b3488b72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:36.907209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-5497b85d-d83b-4dab-8e8c-3e9b3488b72d None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:36.908384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-5497b85d-d83b-4dab-8e8c-3e9b3488b72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:36.974321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 603/1179] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:36:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:36:37.004322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fff2e-e924-4e6c-bbca-d77b367cf851 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:37.042981 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:37.114256 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fff2e-e924-4e6c-bbca-d77b367cf851 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:37.154539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff680f02-1bae-4242-baca-e8e33f07c87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:37.155359 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff680f02-1bae-4242-baca-e8e33f07c87c None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:37.155646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff680f02-1bae-4242-baca-e8e33f07c87c None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:37.155923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff680f02-1bae-4242-baca-e8e33f07c87c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:37.168585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff680f02-1bae-4242-baca-e8e33f07c87c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:36:37.176403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-ff680f02-1bae-4242-baca-e8e33f07c87c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:36:37.176403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:36:37.176403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:36:37.176403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:36:37.176403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:36:37.176403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:36:37.176403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:36:37.176403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:36:37.176403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:36:37.176403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:36:37.176403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:36:37.176403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:36:37.176403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:36:37.176403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:36:37.176403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:36:37.176403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:36:37.177600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:36:37.177600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:36:37.177600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:36:37.177600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:36:37.177600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:36:37.177600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:36:37.177600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:36:37.177600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:36:37.177600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:36:37.177600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:36:37.177600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:36:37.177600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:36:37.177600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:36:37.177600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:36:37.177600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:36:37.177600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:36:37.178711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:36:37.178711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:36:37.178711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:36:37.178711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:36:37.178711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:36:37.178711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:36:37.178711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:36:37.178711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:36:37.178711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:36:37.178711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:36:37.178711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:36:37.178711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:36:37.181828 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 577/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:36:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:36:37.184388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5497b85d-d83b-4dab-8e8c-3e9b3488b72d None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:37.285138 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5497b85d-d83b-4dab-8e8c-3e9b3488b72d None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:37.315406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:37.325480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187425c3-31e6-4e7f-8e63-1689076e00f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:37.326179 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187425c3-31e6-4e7f-8e63-1689076e00f6 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:37.326459 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187425c3-31e6-4e7f-8e63-1689076e00f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:37.326731 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187425c3-31e6-4e7f-8e63-1689076e00f6 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:37.327477 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187425c3-31e6-4e7f-8e63-1689076e00f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:36:37.327780 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187425c3-31e6-4e7f-8e63-1689076e00f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:37.329135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187425c3-31e6-4e7f-8e63-1689076e00f6 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:37.338111 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 604/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:36:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:37.344284 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff680f02-1bae-4242-baca-e8e33f07c87c None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:37.446632 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:37.454345 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff680f02-1bae-4242-baca-e8e33f07c87c None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:37.490893 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570313a0-4ef1-44cf-95f3-32bb2eaf13b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:37.491797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570313a0-4ef1-44cf-95f3-32bb2eaf13b7 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:37.492156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570313a0-4ef1-44cf-95f3-32bb2eaf13b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:37.492433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570313a0-4ef1-44cf-95f3-32bb2eaf13b7 None admin] PATH_INFO: `/v3/users/95ad54b381144095a44d6937a6b2bc89` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:37.506388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570313a0-4ef1-44cf-95f3-32bb2eaf13b7 None admin] RBAC: Authorizing `identity:get_user(user_id=95ad54b381144095a44d6937a6b2bc89)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:37.507990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570313a0-4ef1-44cf-95f3-32bb2eaf13b7 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:37.509676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 578/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:36:37 2022] GET /identity/v3/users/95ad54b381144095a44d6937a6b2bc89 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:37.515659 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187425c3-31e6-4e7f-8e63-1689076e00f6 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:37.643634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187425c3-31e6-4e7f-8e63-1689076e00f6 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:37.684003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d68aa8-02d2-47f1-becc-419f9cf08b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:37.684672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d68aa8-02d2-47f1-becc-419f9cf08b4e None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:37.684960 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d68aa8-02d2-47f1-becc-419f9cf08b4e None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:37.685240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d68aa8-02d2-47f1-becc-419f9cf08b4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:37.748386 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:37.763584 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d68aa8-02d2-47f1-becc-419f9cf08b4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:37.765055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d68aa8-02d2-47f1-becc-419f9cf08b4e None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:37.869398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 605/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:36:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:36:37.875338 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570313a0-4ef1-44cf-95f3-32bb2eaf13b7 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:37.977523 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570313a0-4ef1-44cf-95f3-32bb2eaf13b7 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:38.012544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24048519-c946-4c21-a0b7-1cd1c93c38bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:38.013329 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24048519-c946-4c21-a0b7-1cd1c93c38bd None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:38.013612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24048519-c946-4c21-a0b7-1cd1c93c38bd None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:38.013923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24048519-c946-4c21-a0b7-1cd1c93c38bd None admin] PATH_INFO: `/v3/projects/5b78bc8cc74849bbac5c146a4d076f0a` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:38.015097 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24048519-c946-4c21-a0b7-1cd1c93c38bd None admin] RBAC: Authorizing `identity:get_project(project_id=5b78bc8cc74849bbac5c146a4d076f0a)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:38.016537 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24048519-c946-4c21-a0b7-1cd1c93c38bd None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:38.018266 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 579/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:36:37 2022] GET /identity/v3/projects/5b78bc8cc74849bbac5c146a4d076f0a => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:38.024454 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d68aa8-02d2-47f1-becc-419f9cf08b4e None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:38.110184 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:38.122974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d68aa8-02d2-47f1-becc-419f9cf08b4e None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:38.155820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1eeeb5-5da8-4534-ac7b-781267dc068f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:38.156463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1eeeb5-5da8-4534-ac7b-781267dc068f None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:38.156746 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1eeeb5-5da8-4534-ac7b-781267dc068f None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:38.157019 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1eeeb5-5da8-4534-ac7b-781267dc068f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:38.157866 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1eeeb5-5da8-4534-ac7b-781267dc068f None admin] RBAC: Adding query filter params (role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.project.id=5b78bc8cc74849bbac5c146a4d076f0a, user.id=95ad54b381144095a44d6937a6b2bc89) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:36:38.158176 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1eeeb5-5da8-4534-ac7b-781267dc068f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:38.159604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1eeeb5-5da8-4534-ac7b-781267dc068f None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:38.166294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 606/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:36:38 2022] GET /identity/v3/role_assignments?user.id=95ad54b381144095a44d6937a6b2bc89&scope.project.id=5b78bc8cc74849bbac5c146a4d076f0a&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:40.246026 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b9d81-6035-4592-b8a5-a2a4e6806d8d None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:40.246601 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b9d81-6035-4592-b8a5-a2a4e6806d8d None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:40.246601 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b9d81-6035-4592-b8a5-a2a4e6806d8d None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:40.247988 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 580/1186] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:36:40.253486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec1973-83f9-4a2f-8a58-fdbc4625e548 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:40.253980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec1973-83f9-4a2f-8a58-fdbc4625e548 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:40.254081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec1973-83f9-4a2f-8a58-fdbc4625e548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:40.271403 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-aaec1973-83f9-4a2f-8a58-fdbc4625e548 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:40.273182 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-aaec1973-83f9-4a2f-8a58-fdbc4625e548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:40.341540 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 607/1187] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:36:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:36:40.365422 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b9d81-6035-4592-b8a5-a2a4e6806d8d None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:40.388392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:40.471613 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b9d81-6035-4592-b8a5-a2a4e6806d8d None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:40.511858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b945f2-f15c-4e6f-8949-f3741d5098ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:40.512583 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b945f2-f15c-4e6f-8949-f3741d5098ef None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:40.512869 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b945f2-f15c-4e6f-8949-f3741d5098ef None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:40.513148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b945f2-f15c-4e6f-8949-f3741d5098ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:40.526043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b945f2-f15c-4e6f-8949-f3741d5098ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:36:40.533958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-92b945f2-f15c-4e6f-8949-f3741d5098ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:36:40.533958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:36:40.533958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:36:40.533958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:36:40.533958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:36:40.533958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:36:40.533958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:36:40.533958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:36:40.533958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:36:40.533958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:36:40.533958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:36:40.533958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:36:40.533958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:36:40.533958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:36:40.533958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:36:40.533958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:36:40.535154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:36:40.535154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:36:40.535154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:36:40.535154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:36:40.535154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:36:40.535154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:36:40.535154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:36:40.535154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:36:40.535154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:36:40.535154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:36:40.535154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:36:40.535154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:36:40.535154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:36:40.535154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:36:40.535154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:36:40.535154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:36:40.536181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:36:40.536181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:36:40.536181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:36:40.536181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:36:40.536181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:36:40.536181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:36:40.536181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:36:40.536181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:36:40.536181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:36:40.536181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:36:40.536181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:36:40.536181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:36:40.536181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 581/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:36:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:36:40.543857 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec1973-83f9-4a2f-8a58-fdbc4625e548 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:40.627347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:40.646159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec1973-83f9-4a2f-8a58-fdbc4625e548 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:40.683569 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9a7c2-6529-421e-9765-3693141c428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:40.684261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9a7c2-6529-421e-9765-3693141c428b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:40.684545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9a7c2-6529-421e-9765-3693141c428b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:40.684821 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9a7c2-6529-421e-9765-3693141c428b None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:40.685563 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9a7c2-6529-421e-9765-3693141c428b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:36:40.685906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9a7c2-6529-421e-9765-3693141c428b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:40.687270 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9a7c2-6529-421e-9765-3693141c428b None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:40.696868 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 608/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:36:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:40.702729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b945f2-f15c-4e6f-8949-f3741d5098ef None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:40.786903 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:40.809302 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b945f2-f15c-4e6f-8949-f3741d5098ef None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:40.844463 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf29dbb-8b25-458b-8aeb-e2dfb6174fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:40.845346 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf29dbb-8b25-458b-8aeb-e2dfb6174fd4 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:40.845651 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf29dbb-8b25-458b-8aeb-e2dfb6174fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:40.845934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf29dbb-8b25-458b-8aeb-e2dfb6174fd4 None admin] PATH_INFO: `/v3/users/95ad54b381144095a44d6937a6b2bc89` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:40.847141 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf29dbb-8b25-458b-8aeb-e2dfb6174fd4 None admin] RBAC: Authorizing `identity:get_user(user_id=95ad54b381144095a44d6937a6b2bc89)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:40.848588 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf29dbb-8b25-458b-8aeb-e2dfb6174fd4 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:40.850311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 582/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:36:40 2022] GET /identity/v3/users/95ad54b381144095a44d6937a6b2bc89 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:40.857194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9a7c2-6529-421e-9765-3693141c428b None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:40.963716 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9a7c2-6529-421e-9765-3693141c428b None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:40.997947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc049bd5-791d-4de2-942d-245cf4f46803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:40.998804 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc049bd5-791d-4de2-942d-245cf4f46803 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:40.999085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc049bd5-791d-4de2-942d-245cf4f46803 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:40.999359 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc049bd5-791d-4de2-942d-245cf4f46803 None admin] PATH_INFO: `/v3/projects/5b78bc8cc74849bbac5c146a4d076f0a` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:41.000511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc049bd5-791d-4de2-942d-245cf4f46803 None admin] RBAC: Authorizing `identity:get_project(project_id=5b78bc8cc74849bbac5c146a4d076f0a)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:41.001959 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc049bd5-791d-4de2-942d-245cf4f46803 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:41.003654 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 609/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:36:40 2022] GET /identity/v3/projects/5b78bc8cc74849bbac5c146a4d076f0a => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:41.010352 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf29dbb-8b25-458b-8aeb-e2dfb6174fd4 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:41.109337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf29dbb-8b25-458b-8aeb-e2dfb6174fd4 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:41.143954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1aee2b-51d8-4f01-9eb8-565c51a9b1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:41.144886 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1aee2b-51d8-4f01-9eb8-565c51a9b1d7 None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:41.145171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1aee2b-51d8-4f01-9eb8-565c51a9b1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:41.145445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1aee2b-51d8-4f01-9eb8-565c51a9b1d7 None admin] PATH_INFO: `/v3/projects/5b78bc8cc74849bbac5c146a4d076f0a/users/95ad54b381144095a44d6937a6b2bc89/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:41.148311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1aee2b-51d8-4f01-9eb8-565c51a9b1d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b78bc8cc74849bbac5c146a4d076f0a, user_id=95ad54b381144095a44d6937a6b2bc89, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:41.149795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1aee2b-51d8-4f01-9eb8-565c51a9b1d7 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:41.164779 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 583/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:36:41 2022] PUT /identity/v3/projects/5b78bc8cc74849bbac5c146a4d076f0a/users/95ad54b381144095a44d6937a6b2bc89/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:36:43.209909 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3a897-3a6c-4238-a6ba-5cae4085594e None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:43.210476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3a897-3a6c-4238-a6ba-5cae4085594e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:43.210476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3a897-3a6c-4238-a6ba-5cae4085594e None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:43.211939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 610/1193] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:36:43.217711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf248d1a-613a-4399-9a4e-ee71dc643f3f None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:43.218143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf248d1a-613a-4399-9a4e-ee71dc643f3f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:43.218272 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf248d1a-613a-4399-9a4e-ee71dc643f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:43.235746 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-bf248d1a-613a-4399-9a4e-ee71dc643f3f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:36:43.236940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-bf248d1a-613a-4399-9a4e-ee71dc643f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:36:43.239009 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:43.306074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 584/1194] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:36:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:36:43.329898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3a897-3a6c-4238-a6ba-5cae4085594e None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:43.345891 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:43.434319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3a897-3a6c-4238-a6ba-5cae4085594e None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:43.468073 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21ed78-8e50-4071-94cb-7b8a40abf2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:43.468779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21ed78-8e50-4071-94cb-7b8a40abf2f3 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:43.469056 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21ed78-8e50-4071-94cb-7b8a40abf2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:43.469328 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21ed78-8e50-4071-94cb-7b8a40abf2f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:43.479774 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21ed78-8e50-4071-94cb-7b8a40abf2f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:36:43.487261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-ab21ed78-8e50-4071-94cb-7b8a40abf2f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:36:43.487261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:36:43.487261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:36:43.487261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:36:43.487261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:36:43.487261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:36:43.487261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:36:43.487261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:36:43.487261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:36:43.487261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:36:43.487261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:36:43.487261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:36:43.487261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:36:43.487261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:36:43.487261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:36:43.487261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:36:43.488387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:36:43.488387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:36:43.488387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:36:43.488387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:36:43.488387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:36:43.488387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:36:43.488387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:36:43.488387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:36:43.488387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:36:43.488387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:36:43.488387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:36:43.488387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:36:43.488387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:36:43.488387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:36:43.488387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:36:43.488387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:36:43.489421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:36:43.489421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:36:43.489421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:36:43.489421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:36:43.489421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:36:43.489421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:36:43.489421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:36:43.489421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:36:43.489421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:36:43.489421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:36:43.489421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:36:43.489421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:36:43.489421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 611/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:36:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:36:43.495280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf248d1a-613a-4399-9a4e-ee71dc643f3f None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:43.605192 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf248d1a-613a-4399-9a4e-ee71dc643f3f None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:43.639835 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851a430-d077-4ab8-882a-15ece32fc8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:43.640505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851a430-d077-4ab8-882a-15ece32fc8e7 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:43.640787 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851a430-d077-4ab8-882a-15ece32fc8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:43.641061 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851a430-d077-4ab8-882a-15ece32fc8e7 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:43.641839 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851a430-d077-4ab8-882a-15ece32fc8e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:36:43.642151 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851a430-d077-4ab8-882a-15ece32fc8e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:43.643530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851a430-d077-4ab8-882a-15ece32fc8e7 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:43.649348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:43.653054 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 585/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:36:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:43.658569 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21ed78-8e50-4071-94cb-7b8a40abf2f3 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:43.733954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:43.765134 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21ed78-8e50-4071-94cb-7b8a40abf2f3 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:43.798459 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b0c63-b7d7-4d1e-bb74-ee83986a6daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:43.799331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b0c63-b7d7-4d1e-bb74-ee83986a6daf None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:43.799612 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b0c63-b7d7-4d1e-bb74-ee83986a6daf None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:43.799886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b0c63-b7d7-4d1e-bb74-ee83986a6daf None admin] PATH_INFO: `/v3/users/95ad54b381144095a44d6937a6b2bc89` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:43.801102 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b0c63-b7d7-4d1e-bb74-ee83986a6daf None admin] RBAC: Authorizing `identity:get_user(user_id=95ad54b381144095a44d6937a6b2bc89)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:43.802579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b0c63-b7d7-4d1e-bb74-ee83986a6daf None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:43.805604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 612/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Mon Jun 27 15:36:43 2022] GET /identity/v3/users/95ad54b381144095a44d6937a6b2bc89 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:43.811478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851a430-d077-4ab8-882a-15ece32fc8e7 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:43.912782 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851a430-d077-4ab8-882a-15ece32fc8e7 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:43.947705 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2802629-f4f6-4539-961a-55a61d6d1ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:43.948362 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2802629-f4f6-4539-961a-55a61d6d1ba1 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:43.948646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2802629-f4f6-4539-961a-55a61d6d1ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:43.948924 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2802629-f4f6-4539-961a-55a61d6d1ba1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:44.014780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:44.016879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2802629-f4f6-4539-961a-55a61d6d1ba1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:44.018719 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2802629-f4f6-4539-961a-55a61d6d1ba1 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:44.115654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 586/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:36:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:36:44.122296 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b0c63-b7d7-4d1e-bb74-ee83986a6daf None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:44.224603 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b0c63-b7d7-4d1e-bb74-ee83986a6daf None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:44.259315 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e6dcf-f63c-48bc-ad16-30ea5470cae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:44.260174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e6dcf-f63c-48bc-ad16-30ea5470cae8 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:44.260457 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e6dcf-f63c-48bc-ad16-30ea5470cae8 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:44.260733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e6dcf-f63c-48bc-ad16-30ea5470cae8 None admin] PATH_INFO: `/v3/projects/5b78bc8cc74849bbac5c146a4d076f0a` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:44.261935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e6dcf-f63c-48bc-ad16-30ea5470cae8 None admin] RBAC: Authorizing `identity:get_project(project_id=5b78bc8cc74849bbac5c146a4d076f0a)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:44.263355 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e6dcf-f63c-48bc-ad16-30ea5470cae8 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:44.265099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 613/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Mon Jun 27 15:36:44 2022] GET /identity/v3/projects/5b78bc8cc74849bbac5c146a4d076f0a => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:44.271266 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2802629-f4f6-4539-961a-55a61d6d1ba1 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:36:44.369029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:44.372353 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2802629-f4f6-4539-961a-55a61d6d1ba1 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:36:44.412357 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc764a7-5bb7-4749-b4f5-db34754d1574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:36:44.413664 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc764a7-5bb7-4749-b4f5-db34754d1574 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:36:44.413938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc764a7-5bb7-4749-b4f5-db34754d1574 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:36:44.414211 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc764a7-5bb7-4749-b4f5-db34754d1574 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:36:44.415326 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc764a7-5bb7-4749-b4f5-db34754d1574 None admin] RBAC: Adding query filter params (role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.project.id=5b78bc8cc74849bbac5c146a4d076f0a, user.id=95ad54b381144095a44d6937a6b2bc89) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:36:44.415642 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc764a7-5bb7-4749-b4f5-db34754d1574 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:36:44.417126 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc764a7-5bb7-4749-b4f5-db34754d1574 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:36:44.423951 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 587/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:36:44 2022] GET /identity/v3/role_assignments?user.id=95ad54b381144095a44d6937a6b2bc89&scope.project.id=5b78bc8cc74849bbac5c146a4d076f0a&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:36:54.254625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=96 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:36:54.420997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=18 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:42.463734 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c85e0-434e-42cb-b472-8b72eb73e11e None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:42.464390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c85e0-434e-42cb-b472-8b72eb73e11e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:42.464390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c85e0-434e-42cb-b472-8b72eb73e11e None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:42.467648 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 614/1201] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:37:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:37:42.475146 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efbb77-4a2f-4682-8a1c-86451848fd45 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:42.476035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efbb77-4a2f-4682-8a1c-86451848fd45 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:42.476035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8efbb77-4a2f-4682-8a1c-86451848fd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:42.502030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-e8efbb77-4a2f-4682-8a1c-86451848fd45 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:37:42.504092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e8efbb77-4a2f-4682-8a1c-86451848fd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:37:42.611256 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 588/1202] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:37:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:37:42.638455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c85e0-434e-42cb-b472-8b72eb73e11e None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:42.778766 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c85e0-434e-42cb-b472-8b72eb73e11e None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:42.825036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef05f2c-2623-42bc-ba65-c2ca4bfc0eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:42.826210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef05f2c-2623-42bc-ba65-c2ca4bfc0eb0 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:42.826548 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef05f2c-2623-42bc-ba65-c2ca4bfc0eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:42.826827 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef05f2c-2623-42bc-ba65-c2ca4bfc0eb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:42.842883 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef05f2c-2623-42bc-ba65-c2ca4bfc0eb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:42.844463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef05f2c-2623-42bc-ba65-c2ca4bfc0eb0 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:42.872595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-cef05f2c-2623-42bc-ba65-c2ca4bfc0eb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:37:42.872595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:37:42.872595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:37:42.872595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:37:42.872595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:37:42.872595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:37:42.872595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:37:42.872595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:37:42.872595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:37:42.872595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:37:42.872595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:37:42.872595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:37:42.872595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:37:42.872595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:37:42.872595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:37:42.873713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:37:42.873713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:37:42.873713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:37:42.873713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:37:42.873713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:37:42.873713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:37:42.873713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:37:42.873713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:37:42.873713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:37:42.873713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:37:42.873713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:37:42.873713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:37:42.873713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:37:42.873713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:37:42.873713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:37:42.873713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:37:42.873713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:37:42.874689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:37:42.874689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:37:42.874689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:37:42.874689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:37:42.874689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 615/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:37:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:37:42.880597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efbb77-4a2f-4682-8a1c-86451848fd45 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:43.000624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8efbb77-4a2f-4682-8a1c-86451848fd45 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:43.042088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:43.048782 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c618d-8d2e-43b2-9d35-9e35fa69395e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:43.049709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c618d-8d2e-43b2-9d35-9e35fa69395e None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:43.049932 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c618d-8d2e-43b2-9d35-9e35fa69395e None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:43.050102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c618d-8d2e-43b2-9d35-9e35fa69395e None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:43.050983 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c618d-8d2e-43b2-9d35-9e35fa69395e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:37:43.051210 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c618d-8d2e-43b2-9d35-9e35fa69395e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:43.053353 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c618d-8d2e-43b2-9d35-9e35fa69395e None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:43.066807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 589/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:37:43.073665 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef05f2c-2623-42bc-ba65-c2ca4bfc0eb0 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:43.155466 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:43.217909 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef05f2c-2623-42bc-ba65-c2ca4bfc0eb0 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:43.262310 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8613b-33d6-49e0-9afd-b2e6caf92cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:43.262972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8613b-33d6-49e0-9afd-b2e6caf92cbf None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:43.263233 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8613b-33d6-49e0-9afd-b2e6caf92cbf None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:43.263503 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8613b-33d6-49e0-9afd-b2e6caf92cbf None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:43.264205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8613b-33d6-49e0-9afd-b2e6caf92cbf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:43.265673 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8613b-33d6-49e0-9afd-b2e6caf92cbf None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:43.289854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 616/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Mon Jun 27 15:37:43 2022] POST /identity/v3/users => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:37:45.606780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8fd18-a2bf-49d2-b1ad-9e5e3250c812 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:45.607336 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8fd18-a2bf-49d2-b1ad-9e5e3250c812 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:45.607437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8fd18-a2bf-49d2-b1ad-9e5e3250c812 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:45.608779 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 590/1206] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:37:45.616093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5444a65-6f54-4ba5-be0e-0390b5050691 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:45.616685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5444a65-6f54-4ba5-be0e-0390b5050691 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:45.617212 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5444a65-6f54-4ba5-be0e-0390b5050691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:45.639305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-b5444a65-6f54-4ba5-be0e-0390b5050691 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:37:45.640538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-b5444a65-6f54-4ba5-be0e-0390b5050691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:37:45.749676 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 617/1207] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:37:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:37:45.782294 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8fd18-a2bf-49d2-b1ad-9e5e3250c812 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:45.934823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8fd18-a2bf-49d2-b1ad-9e5e3250c812 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:45.995252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb76228-72b3-4e57-b421-cb88075eaf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:45.996095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb76228-72b3-4e57-b421-cb88075eaf34 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:45.996934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb76228-72b3-4e57-b421-cb88075eaf34 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:45.996934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb76228-72b3-4e57-b421-cb88075eaf34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:46.010067 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb76228-72b3-4e57-b421-cb88075eaf34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:37:46.018096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-1eb76228-72b3-4e57-b421-cb88075eaf34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:37:46.018096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:37:46.018096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:37:46.018096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:37:46.018096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:37:46.018096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:37:46.018096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:37:46.018096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:37:46.018096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:37:46.018096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:37:46.018096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:37:46.018096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:37:46.018096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:37:46.018096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:37:46.018096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:37:46.018096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:37:46.019204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:37:46.019204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:37:46.019204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:37:46.019204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:37:46.019204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:37:46.019204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:37:46.019204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:37:46.019204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:37:46.019204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:37:46.019204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:37:46.019204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:37:46.019204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:37:46.019204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:37:46.019204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:37:46.019204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:37:46.019204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:37:46.022628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:37:46.022628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:37:46.022628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:37:46.022628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:37:46.022628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:37:46.022628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:37:46.022628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:37:46.022628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:37:46.022628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:37:46.022628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:37:46.022628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:37:46.022628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:37:46.022628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 591/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:37:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:37:46.026340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5444a65-6f54-4ba5-be0e-0390b5050691 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:46.082591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:46.172414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5444a65-6f54-4ba5-be0e-0390b5050691 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:46.233205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa688829-fcb4-48a4-9dbe-b0cb8d2fa729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:46.233954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa688829-fcb4-48a4-9dbe-b0cb8d2fa729 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:46.236015 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa688829-fcb4-48a4-9dbe-b0cb8d2fa729 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:46.236015 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa688829-fcb4-48a4-9dbe-b0cb8d2fa729 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:46.236015 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa688829-fcb4-48a4-9dbe-b0cb8d2fa729 None admin] RBAC: Adding query filter params (name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:37:46.236831 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa688829-fcb4-48a4-9dbe-b0cb8d2fa729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:46.238199 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa688829-fcb4-48a4-9dbe-b0cb8d2fa729 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:46.256445 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 618/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:37:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:37:46.262464 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb76228-72b3-4e57-b421-cb88075eaf34 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:46.327994 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:46.414358 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb76228-72b3-4e57-b421-cb88075eaf34 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:46.465252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44331b9c-e234-4740-94f7-1624d18c1e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:46.466239 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44331b9c-e234-4740-94f7-1624d18c1e90 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:46.466519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44331b9c-e234-4740-94f7-1624d18c1e90 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:46.466838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44331b9c-e234-4740-94f7-1624d18c1e90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:46.479058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44331b9c-e234-4740-94f7-1624d18c1e90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:46.480678 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44331b9c-e234-4740-94f7-1624d18c1e90 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:46.504125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-44331b9c-e234-4740-94f7-1624d18c1e90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:37:46.504125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:37:46.504125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:37:46.504125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:37:46.504125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:37:46.504125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:37:46.504125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:37:46.504125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:37:46.504125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:37:46.504125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:37:46.504125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:37:46.504125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:37:46.504125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:37:46.504125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:37:46.504125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:37:46.510038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:37:46.510038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:37:46.510038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:37:46.510038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:37:46.510038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:37:46.510038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:37:46.510038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:37:46.510038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:37:46.510038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:37:46.510038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:37:46.510038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:37:46.510038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:37:46.510038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:37:46.510038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:37:46.510038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:37:46.510038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:37:46.510038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:37:46.511127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:37:46.511127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:37:46.511127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:37:46.511127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:37:46.511127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 592/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:37:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:37:46.511463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa688829-fcb4-48a4-9dbe-b0cb8d2fa729 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:46.621978 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:46.625855 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa688829-fcb4-48a4-9dbe-b0cb8d2fa729 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:46.663970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c2a46-303c-4d64-84b0-6013056a88bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:46.664641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c2a46-303c-4d64-84b0-6013056a88bd None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:46.665175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c2a46-303c-4d64-84b0-6013056a88bd None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:46.665454 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c2a46-303c-4d64-84b0-6013056a88bd None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:46.666200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c2a46-303c-4d64-84b0-6013056a88bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:37:46.666504 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c2a46-303c-4d64-84b0-6013056a88bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:46.667862 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c2a46-303c-4d64-84b0-6013056a88bd None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:46.680837 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 619/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:37:46.686046 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44331b9c-e234-4740-94f7-1624d18c1e90 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:46.809988 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44331b9c-e234-4740-94f7-1624d18c1e90 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:46.811819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:46.857432 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70824f34-a2dd-4f5d-a052-68883e8e4e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:46.858321 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70824f34-a2dd-4f5d-a052-68883e8e4e1b None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:46.858609 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70824f34-a2dd-4f5d-a052-68883e8e4e1b None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:46.858934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70824f34-a2dd-4f5d-a052-68883e8e4e1b None admin] PATH_INFO: `/v3/users/placement` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:46.867717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70824f34-a2dd-4f5d-a052-68883e8e4e1b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:46.869357 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70824f34-a2dd-4f5d-a052-68883e8e4e1b None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:46.887113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-70824f34-a2dd-4f5d-a052-68883e8e4e1b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:37:46.887113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:37:46.887113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:37:46.887113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:37:46.887113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:37:46.887113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:37:46.887113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:37:46.887113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:37:46.887113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:37:46.887113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:37:46.887113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:37:46.887113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:37:46.887113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:37:46.887113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:37:46.887113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:37:46.887113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:37:46.888593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:37:46.888593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:37:46.888593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:37:46.888593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:37:46.888593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:37:46.888593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:37:46.888593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:37:46.888593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:37:46.888593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:37:46.888593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:37:46.888593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:37:46.888593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:37:46.888593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:37:46.888593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:37:46.888593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:37:46.888593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:37:46.888593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:37:46.888593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:37:46.889734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:37:46.889734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:37:46.889734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:37:46.889734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:37:46.889734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:37:46.889734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:37:46.889734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:37:46.889734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:37:46.889734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:37:46.889734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:37:46.889734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 593/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:37:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:37:46.894347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c2a46-303c-4d64-84b0-6013056a88bd None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:47.029487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c2a46-303c-4d64-84b0-6013056a88bd None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:47.077005 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd263709-c1e2-4f03-bf91-b737db585fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:47.077609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd263709-c1e2-4f03-bf91-b737db585fd7 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:47.077943 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd263709-c1e2-4f03-bf91-b737db585fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:47.078293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd263709-c1e2-4f03-bf91-b737db585fd7 None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:47.079267 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd263709-c1e2-4f03-bf91-b737db585fd7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:37:47.079591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd263709-c1e2-4f03-bf91-b737db585fd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:47.081026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd263709-c1e2-4f03-bf91-b737db585fd7 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:47.103157 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 620/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Jun 27 15:37:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:37:47.108995 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70824f34-a2dd-4f5d-a052-68883e8e4e1b None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:47.238685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70824f34-a2dd-4f5d-a052-68883e8e4e1b None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:47.290264 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c56e8-c36a-474a-94bd-f922c8ea4cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:47.291299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c56e8-c36a-474a-94bd-f922c8ea4cc7 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:47.291662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c56e8-c36a-474a-94bd-f922c8ea4cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:47.291962 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c56e8-c36a-474a-94bd-f922c8ea4cc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:47.331190 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:47.390075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565c56e8-c36a-474a-94bd-f922c8ea4cc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:47.392105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565c56e8-c36a-474a-94bd-f922c8ea4cc7 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:47.525994 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 594/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:37:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:37:47.532400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd263709-c1e2-4f03-bf91-b737db585fd7 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:47.560654 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:47.682591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd263709-c1e2-4f03-bf91-b737db585fd7 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:47.729522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5014ec6-1a4a-401e-bdcd-c0a258a31092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:47.730418 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5014ec6-1a4a-401e-bdcd-c0a258a31092 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:47.730684 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5014ec6-1a4a-401e-bdcd-c0a258a31092 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:47.731170 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5014ec6-1a4a-401e-bdcd-c0a258a31092 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:47.748300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5014ec6-1a4a-401e-bdcd-c0a258a31092 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:47.752417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5014ec6-1a4a-401e-bdcd-c0a258a31092 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:47.778958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-f5014ec6-1a4a-401e-bdcd-c0a258a31092 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:37:47.778958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:37:47.778958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:37:47.778958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:37:47.778958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:37:47.778958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:37:47.778958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:37:47.778958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:37:47.778958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:37:47.778958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:37:47.778958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:37:47.778958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:37:47.778958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:37:47.778958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:37:47.778958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:37:47.780710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:37:47.780710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:37:47.780710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:37:47.780710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:37:47.780710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:37:47.780710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:37:47.780710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:37:47.780710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:37:47.780710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:37:47.780710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:37:47.780710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:37:47.780710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:37:47.780710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:37:47.780710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:37:47.780710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:37:47.780710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:37:47.780710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:37:47.781874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:37:47.781874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:37:47.781874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:37:47.781874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:37:47.781874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 621/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:37:47.789069 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c56e8-c36a-474a-94bd-f922c8ea4cc7 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:47.867889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:47.916001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c56e8-c36a-474a-94bd-f922c8ea4cc7 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:47.955602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5c91f-21ca-49d0-9b16-7aec2949b541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:47.956204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5c91f-21ca-49d0-9b16-7aec2949b541 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:47.956521 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5c91f-21ca-49d0-9b16-7aec2949b541 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:47.956762 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5c91f-21ca-49d0-9b16-7aec2949b541 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:47.957495 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5c91f-21ca-49d0-9b16-7aec2949b541 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:37:47.958488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5c91f-21ca-49d0-9b16-7aec2949b541 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:47.960031 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5c91f-21ca-49d0-9b16-7aec2949b541 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:47.977694 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 595/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:37:47.978996 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5014ec6-1a4a-401e-bdcd-c0a258a31092 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:48.055530 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:48.094383 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5014ec6-1a4a-401e-bdcd-c0a258a31092 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:48.139734 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618a7415-ce57-4855-a725-770b04367789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:48.140527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a7415-ce57-4855-a725-770b04367789 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:48.140876 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a7415-ce57-4855-a725-770b04367789 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:48.141084 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a7415-ce57-4855-a725-770b04367789 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:48.150442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618a7415-ce57-4855-a725-770b04367789 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:48.151907 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618a7415-ce57-4855-a725-770b04367789 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:48.166686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-618a7415-ce57-4855-a725-770b04367789 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:37:48.166686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:37:48.166686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:37:48.166686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:37:48.166686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:37:48.166686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:37:48.166686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:37:48.166686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:37:48.166686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:37:48.166686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:37:48.166686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:37:48.166686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:37:48.166686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:37:48.166686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:37:48.166686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:37:48.167685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:37:48.167685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:37:48.167685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:37:48.167685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:37:48.167685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:37:48.167685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:37:48.167685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:37:48.167685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:37:48.167685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:37:48.167685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:37:48.167685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:37:48.167685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:37:48.167685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:37:48.167685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:37:48.167685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:37:48.167685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:37:48.167685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:37:48.168591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:37:48.168591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:37:48.168591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:37:48.168591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:37:48.168591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:37:48.168591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:37:48.168591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:37:48.168591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:37:48.168591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 622/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:37:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:37:48.173834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5c91f-21ca-49d0-9b16-7aec2949b541 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:48.310023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5c91f-21ca-49d0-9b16-7aec2949b541 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:48.346748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:48.362450 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201431e-291b-446b-a617-5c2d04ad21aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:48.363371 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201431e-291b-446b-a617-5c2d04ad21aa None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:48.363838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201431e-291b-446b-a617-5c2d04ad21aa None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:48.364828 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201431e-291b-446b-a617-5c2d04ad21aa None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:48.365822 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201431e-291b-446b-a617-5c2d04ad21aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:37:48.366287 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201431e-291b-446b-a617-5c2d04ad21aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:48.368329 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201431e-291b-446b-a617-5c2d04ad21aa None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:48.389199 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 596/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:37:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:37:48.396578 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618a7415-ce57-4855-a725-770b04367789 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:48.521076 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618a7415-ce57-4855-a725-770b04367789 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:48.556204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:48.564347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bba813-2934-430a-bbf1-a74a3abd1486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:48.565286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bba813-2934-430a-bbf1-a74a3abd1486 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:48.566007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bba813-2934-430a-bbf1-a74a3abd1486 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:48.566532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bba813-2934-430a-bbf1-a74a3abd1486 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:48.567919 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bba813-2934-430a-bbf1-a74a3abd1486 None admin] RBAC: Adding query filter params (role.id=805851682adf4debbe99b1681c9fcdfc, scope.project.id=7be83fb6f20341139147373837320148, user.id=ffafb68f1bb941588a5217ad83ad907c) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:37:48.568426 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bba813-2934-430a-bbf1-a74a3abd1486 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:48.570123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bba813-2934-430a-bbf1-a74a3abd1486 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:48.576622 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 623/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:37:48 2022] GET /identity/v3/role_assignments?user.id=ffafb68f1bb941588a5217ad83ad907c&scope.project.id=7be83fb6f20341139147373837320148&role.id=805851682adf4debbe99b1681c9fcdfc => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:37:51.130554 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fd314-6ba7-4bc4-b12d-a8d1516522cc None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:51.131609 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fd314-6ba7-4bc4-b12d-a8d1516522cc None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:51.131609 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fd314-6ba7-4bc4-b12d-a8d1516522cc None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:51.133204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 597/1220] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:37:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:37:51.142437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468f8f8-8f8a-4a19-9c99-d5cb1291380f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:51.143019 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468f8f8-8f8a-4a19-9c99-d5cb1291380f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:51.143349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468f8f8-8f8a-4a19-9c99-d5cb1291380f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:51.172184 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-5468f8f8-8f8a-4a19-9c99-d5cb1291380f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:37:51.173963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-5468f8f8-8f8a-4a19-9c99-d5cb1291380f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:37:51.281932 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 624/1221] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:37:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:37:51.317107 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fd314-6ba7-4bc4-b12d-a8d1516522cc None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:51.492360 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fd314-6ba7-4bc4-b12d-a8d1516522cc None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:51.548333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b2eb2-9b94-46d7-92b3-ff1f925fb8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:51.549483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b2eb2-9b94-46d7-92b3-ff1f925fb8cf None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:51.549959 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b2eb2-9b94-46d7-92b3-ff1f925fb8cf None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:51.550539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b2eb2-9b94-46d7-92b3-ff1f925fb8cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:51.568197 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b2eb2-9b94-46d7-92b3-ff1f925fb8cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:37:51.580896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-b91b2eb2-9b94-46d7-92b3-ff1f925fb8cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:37:51.580896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:37:51.580896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:37:51.580896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:37:51.580896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:37:51.580896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:37:51.580896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:37:51.580896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:37:51.580896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:37:51.580896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:37:51.580896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:37:51.580896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:37:51.580896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:37:51.580896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:37:51.580896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:37:51.580896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:37:51.583143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:37:51.583143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:37:51.583143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:37:51.583143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:37:51.583143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:37:51.583143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:37:51.583143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:37:51.583143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:37:51.583143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:37:51.583143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:37:51.583143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:37:51.583143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:37:51.583143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:37:51.583143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:37:51.583143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:37:51.583143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:37:51.585074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:37:51.585074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:37:51.585074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:37:51.585074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:37:51.585074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:37:51.585074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:37:51.585074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:37:51.585074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:37:51.585074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:37:51.585074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:37:51.585074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:37:51.585074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:37:51.585074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 598/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:37:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:37:51.591017 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468f8f8-8f8a-4a19-9c99-d5cb1291380f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:51.733479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468f8f8-8f8a-4a19-9c99-d5cb1291380f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:51.776035 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:51.790904 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3b59f-66c9-45b6-9917-41627c51dfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:51.791587 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3b59f-66c9-45b6-9917-41627c51dfd6 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:51.791875 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3b59f-66c9-45b6-9917-41627c51dfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:51.792154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3b59f-66c9-45b6-9917-41627c51dfd6 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:51.792917 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3b59f-66c9-45b6-9917-41627c51dfd6 None admin] RBAC: Adding query filter params (name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:37:51.793227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3b59f-66c9-45b6-9917-41627c51dfd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:51.794664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3b59f-66c9-45b6-9917-41627c51dfd6 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:51.804851 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 625/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:37:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:37:51.811432 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b2eb2-9b94-46d7-92b3-ff1f925fb8cf None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:51.878071 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:51.949224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b2eb2-9b94-46d7-92b3-ff1f925fb8cf None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:51.994729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe89a4-6e8b-4702-8267-647c34d0d72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:51.995619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe89a4-6e8b-4702-8267-647c34d0d72a None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:51.995913 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe89a4-6e8b-4702-8267-647c34d0d72a None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:51.996192 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe89a4-6e8b-4702-8267-647c34d0d72a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:52.009064 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe89a4-6e8b-4702-8267-647c34d0d72a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:52.010597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe89a4-6e8b-4702-8267-647c34d0d72a None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:52.043534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-4ffe89a4-6e8b-4702-8267-647c34d0d72a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:37:52.043534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:37:52.043534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:37:52.043534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:37:52.043534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:37:52.043534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:37:52.043534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:37:52.043534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:37:52.043534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:37:52.043534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:37:52.043534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:37:52.043534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:37:52.043534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:37:52.043534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:37:52.043534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:37:52.044949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:37:52.044949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:37:52.044949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:37:52.044949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:37:52.044949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:37:52.044949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:37:52.044949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:37:52.044949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:37:52.044949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:37:52.044949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:37:52.044949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:37:52.044949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:37:52.044949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:37:52.044949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:37:52.044949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:37:52.044949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:37:52.044949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:37:52.046605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:37:52.046605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:37:52.046605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:37:52.046605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:37:52.046605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 599/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:37:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:37:52.052768 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3b59f-66c9-45b6-9917-41627c51dfd6 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:52.196938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3b59f-66c9-45b6-9917-41627c51dfd6 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:52.236777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5928e4d-3292-46b7-99f3-9768e00d2c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:52.237527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5928e4d-3292-46b7-99f3-9768e00d2c30 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:52.237792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5928e4d-3292-46b7-99f3-9768e00d2c30 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:52.238185 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5928e4d-3292-46b7-99f3-9768e00d2c30 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:52.239010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5928e4d-3292-46b7-99f3-9768e00d2c30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:37:52.239825 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5928e4d-3292-46b7-99f3-9768e00d2c30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:52.241430 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5928e4d-3292-46b7-99f3-9768e00d2c30 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:52.256595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 626/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:37:52.265312 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe89a4-6e8b-4702-8267-647c34d0d72a None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:52.415821 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:52.423218 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe89a4-6e8b-4702-8267-647c34d0d72a None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:52.472099 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4ba49-0960-4791-8bb6-ea6edad01247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:52.472918 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4ba49-0960-4791-8bb6-ea6edad01247 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:52.473195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4ba49-0960-4791-8bb6-ea6edad01247 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:52.473606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4ba49-0960-4791-8bb6-ea6edad01247 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:52.482566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4ba49-0960-4791-8bb6-ea6edad01247 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:52.484110 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4ba49-0960-4791-8bb6-ea6edad01247 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:52.500508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-eaf4ba49-0960-4791-8bb6-ea6edad01247 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:37:52.500508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:37:52.500508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:37:52.500508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:37:52.500508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:37:52.500508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:37:52.500508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:37:52.500508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:37:52.500508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:37:52.500508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:37:52.500508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:37:52.500508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:37:52.500508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:37:52.500508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:37:52.500508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:37:52.500508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:37:52.502832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:37:52.502832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:37:52.502832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:37:52.502832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:37:52.502832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:37:52.502832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:37:52.502832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:37:52.502832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:37:52.502832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:37:52.502832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:37:52.502832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:37:52.502832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:37:52.502832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:37:52.502832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:37:52.502832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:37:52.502832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:37:52.502832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:37:52.502832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:37:52.504456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:37:52.504456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:37:52.504456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:37:52.504456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:37:52.504456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:37:52.504456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:37:52.504456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:37:52.504456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:37:52.504456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:37:52.504456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:37:52.504456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 600/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:37:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:37:52.509047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5928e4d-3292-46b7-99f3-9768e00d2c30 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:52.566686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:52.652262 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5928e4d-3292-46b7-99f3-9768e00d2c30 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:52.692435 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d4807-ac2b-4093-b0a9-31d51ada5f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:52.693079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d4807-ac2b-4093-b0a9-31d51ada5f2d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:52.693380 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d4807-ac2b-4093-b0a9-31d51ada5f2d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:52.693680 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d4807-ac2b-4093-b0a9-31d51ada5f2d None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:52.694544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d4807-ac2b-4093-b0a9-31d51ada5f2d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:37:52.694863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d4807-ac2b-4093-b0a9-31d51ada5f2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:52.696329 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d4807-ac2b-4093-b0a9-31d51ada5f2d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:52.717498 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 627/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Jun 27 15:37:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:37:52.727009 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4ba49-0960-4791-8bb6-ea6edad01247 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:52.864569 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4ba49-0960-4791-8bb6-ea6edad01247 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:52.926278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17476e0-732c-4a83-be28-260416960153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:52.927661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17476e0-732c-4a83-be28-260416960153 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:52.927940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17476e0-732c-4a83-be28-260416960153 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:52.928230 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17476e0-732c-4a83-be28-260416960153 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:52.944627 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17476e0-732c-4a83-be28-260416960153 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:52.946104 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17476e0-732c-4a83-be28-260416960153 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:52.970895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-c17476e0-732c-4a83-be28-260416960153 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:37:52.970895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:37:52.970895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:37:52.970895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:37:52.970895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:37:52.970895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:37:52.970895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:37:52.970895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:37:52.970895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:37:52.970895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:37:52.970895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:37:52.970895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:37:52.970895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:37:52.970895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:37:52.970895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:37:52.972090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:37:52.972090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:37:52.972090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:37:52.972090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:37:52.972090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:37:52.972090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:37:52.972090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:37:52.972090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:37:52.972090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:37:52.972090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:37:52.972090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:37:52.972090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:37:52.972090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:37:52.972090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:37:52.972090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:37:52.972090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:37:52.972090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:37:52.973033 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:37:52.973033 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:37:52.973033 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:37:52.973033 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:37:52.973033 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 601/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:37:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:37:52.978506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d4807-ac2b-4093-b0a9-31d51ada5f2d None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:53.115271 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d4807-ac2b-4093-b0a9-31d51ada5f2d None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:53.119716 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:53.170588 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df69ab0-e13a-4c9c-95a1-561e0a6ac0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:53.171449 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df69ab0-e13a-4c9c-95a1-561e0a6ac0a0 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:53.171913 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df69ab0-e13a-4c9c-95a1-561e0a6ac0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:53.172240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df69ab0-e13a-4c9c-95a1-561e0a6ac0a0 None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:53.173293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df69ab0-e13a-4c9c-95a1-561e0a6ac0a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:37:53.173710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df69ab0-e13a-4c9c-95a1-561e0a6ac0a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:53.175292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df69ab0-e13a-4c9c-95a1-561e0a6ac0a0 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:53.189755 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 628/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:37:53.197129 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17476e0-732c-4a83-be28-260416960153 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:53.210234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:53.325369 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17476e0-732c-4a83-be28-260416960153 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:53.372290 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c27df-20e3-47f0-8cd8-6ee2623fbfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:53.373135 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c27df-20e3-47f0-8cd8-6ee2623fbfb7 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:53.373426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c27df-20e3-47f0-8cd8-6ee2623fbfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:53.373789 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c27df-20e3-47f0-8cd8-6ee2623fbfb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:53.383320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c27df-20e3-47f0-8cd8-6ee2623fbfb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:53.384858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c27df-20e3-47f0-8cd8-6ee2623fbfb7 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:53.407223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-912c27df-20e3-47f0-8cd8-6ee2623fbfb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:37:53.407223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:37:53.407223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:37:53.407223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:37:53.407223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:37:53.407223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:37:53.407223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:37:53.407223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:37:53.407223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:37:53.407223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:37:53.407223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:37:53.407223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:37:53.407223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:37:53.407223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:37:53.407223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:37:53.409292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:37:53.409292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:37:53.409292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:37:53.409292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:37:53.409292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:37:53.409292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:37:53.409292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:37:53.409292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:37:53.409292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:37:53.409292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:37:53.409292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:37:53.409292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:37:53.409292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:37:53.409292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:37:53.409292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:37:53.409292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:37:53.409292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:37:53.410224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:37:53.410224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:37:53.410224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:37:53.410224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:37:53.410224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:37:53.410224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:37:53.410224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:37:53.410224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:37:53.410224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 602/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:37:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:37:53.415509 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df69ab0-e13a-4c9c-95a1-561e0a6ac0a0 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:53.548764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df69ab0-e13a-4c9c-95a1-561e0a6ac0a0 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:53.600531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266b943-906f-4090-b873-065493c57248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:53.601230 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266b943-906f-4090-b873-065493c57248 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:53.601522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266b943-906f-4090-b873-065493c57248 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:53.601981 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266b943-906f-4090-b873-065493c57248 None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:53.602821 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266b943-906f-4090-b873-065493c57248 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:37:53.603137 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266b943-906f-4090-b873-065493c57248 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:53.604652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266b943-906f-4090-b873-065493c57248 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:53.626268 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 629/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:37:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:37:53.634138 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c27df-20e3-47f0-8cd8-6ee2623fbfb7 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:53.747071 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:53.803718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c27df-20e3-47f0-8cd8-6ee2623fbfb7 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:53.862480 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f60a0-e05f-4738-8cc1-6af9eb7df270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:53.864077 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f60a0-e05f-4738-8cc1-6af9eb7df270 None admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:53.865446 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f60a0-e05f-4738-8cc1-6af9eb7df270 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:53.865965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f60a0-e05f-4738-8cc1-6af9eb7df270 None admin] PATH_INFO: `/v3/projects/7be83fb6f20341139147373837320148/users/ffafb68f1bb941588a5217ad83ad907c/roles/805851682adf4debbe99b1681c9fcdfc` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:53.907523 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f60a0-e05f-4738-8cc1-6af9eb7df270 None admin] RBAC: Authorizing `identity:create_grant(project_id=7be83fb6f20341139147373837320148, user_id=ffafb68f1bb941588a5217ad83ad907c, role_id=805851682adf4debbe99b1681c9fcdfc)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:53.912573 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f60a0-e05f-4738-8cc1-6af9eb7df270 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:53.931286 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 603/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:37:53 2022] PUT /identity/v3/projects/7be83fb6f20341139147373837320148/users/ffafb68f1bb941588a5217ad83ad907c/roles/805851682adf4debbe99b1681c9fcdfc => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:37:56.554380 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf673c35-71fd-43e0-ad48-78cbca428e3e None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:56.554990 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf673c35-71fd-43e0-ad48-78cbca428e3e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:56.554990 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf673c35-71fd-43e0-ad48-78cbca428e3e None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:56.556382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 630/1233] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:37:56.564646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957c32e-9676-4dc9-8021-5e96d493ac56 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:56.565192 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957c32e-9676-4dc9-8021-5e96d493ac56 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:56.565690 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957c32e-9676-4dc9-8021-5e96d493ac56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:56.588070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-d957c32e-9676-4dc9-8021-5e96d493ac56 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:37:56.589604 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d957c32e-9676-4dc9-8021-5e96d493ac56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:37:56.675415 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 604/1234] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:37:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:37:56.709313 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf673c35-71fd-43e0-ad48-78cbca428e3e None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:56.742649 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:56.854166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf673c35-71fd-43e0-ad48-78cbca428e3e None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:56.906683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070845f-e62b-4332-b228-62317628e355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:56.907497 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070845f-e62b-4332-b228-62317628e355 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:56.907783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070845f-e62b-4332-b228-62317628e355 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:56.908048 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070845f-e62b-4332-b228-62317628e355 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:56.922592 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070845f-e62b-4332-b228-62317628e355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:37:56.931748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-c070845f-e62b-4332-b228-62317628e355 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:37:56.931748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:37:56.931748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:37:56.931748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:37:56.931748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:37:56.931748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:37:56.931748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:37:56.931748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:37:56.931748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:37:56.931748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:37:56.931748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:37:56.931748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:37:56.931748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:37:56.931748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:37:56.931748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:37:56.931748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:37:56.933000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:37:56.933000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:37:56.933000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:37:56.933000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:37:56.933000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:37:56.933000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:37:56.933000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:37:56.933000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:37:56.933000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:37:56.933000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:37:56.933000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:37:56.933000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:37:56.933000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:37:56.933000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:37:56.933000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:37:56.933000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:37:56.934036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:37:56.934036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:37:56.934036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:37:56.934036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:37:56.934036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:37:56.934036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:37:56.934036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:37:56.934036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:37:56.934036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:37:56.934036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:37:56.934036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 15:37:56.934036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:37:56.934036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 631/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Mon Jun 27 15:37:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:37:56.939279 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957c32e-9676-4dc9-8021-5e96d493ac56 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:57.012132 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:57.077082 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957c32e-9676-4dc9-8021-5e96d493ac56 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:57.126911 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acee5e-8cff-453e-99aa-af39defac98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:57.127902 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acee5e-8cff-453e-99aa-af39defac98b None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:57.128222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acee5e-8cff-453e-99aa-af39defac98b None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:57.128704 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acee5e-8cff-453e-99aa-af39defac98b None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:57.129979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0acee5e-8cff-453e-99aa-af39defac98b None admin] RBAC: Adding query filter params (name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:37:57.130796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0acee5e-8cff-453e-99aa-af39defac98b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:57.132477 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0acee5e-8cff-453e-99aa-af39defac98b None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:57.144658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 605/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Mon Jun 27 15:37:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:37:57.151814 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070845f-e62b-4332-b228-62317628e355 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:57.250047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:57.272213 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070845f-e62b-4332-b228-62317628e355 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:57.315865 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df5777-6da7-4646-86e4-72d4c23ef374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:57.316847 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df5777-6da7-4646-86e4-72d4c23ef374 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:57.317866 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df5777-6da7-4646-86e4-72d4c23ef374 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:57.318455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df5777-6da7-4646-86e4-72d4c23ef374 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:57.334446 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9df5777-6da7-4646-86e4-72d4c23ef374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:57.336160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9df5777-6da7-4646-86e4-72d4c23ef374 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:57.368718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-e9df5777-6da7-4646-86e4-72d4c23ef374 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:37:57.368718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:37:57.368718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:37:57.368718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:37:57.368718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:37:57.368718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:37:57.368718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:37:57.368718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:37:57.368718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:37:57.368718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:37:57.368718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:37:57.368718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:37:57.368718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:37:57.368718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:37:57.368718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:37:57.370049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:37:57.370049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:37:57.370049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:37:57.370049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:37:57.370049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:37:57.370049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:37:57.370049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:37:57.370049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:37:57.370049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:37:57.370049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:37:57.370049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:37:57.370049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:37:57.370049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:37:57.370049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:37:57.370049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:37:57.370049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:37:57.370049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:37:57.371129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:37:57.371129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:37:57.371129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:37:57.371129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:37:57.371595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 632/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:37:57.377661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acee5e-8cff-453e-99aa-af39defac98b None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:57.516058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acee5e-8cff-453e-99aa-af39defac98b None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:57.528713 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:57.575191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3cc1d-229b-4d12-8680-a232c21250af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:57.576101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3cc1d-229b-4d12-8680-a232c21250af None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:57.576473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3cc1d-229b-4d12-8680-a232c21250af None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:57.576893 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3cc1d-229b-4d12-8680-a232c21250af None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:57.578158 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3cc1d-229b-4d12-8680-a232c21250af None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:37:57.578693 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3cc1d-229b-4d12-8680-a232c21250af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:57.581344 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3cc1d-229b-4d12-8680-a232c21250af None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:57.603853 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 606/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:37:57.611425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df5777-6da7-4646-86e4-72d4c23ef374 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:57.775834 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df5777-6da7-4646-86e4-72d4c23ef374 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:57.850180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7ae2e-ce30-4403-bb31-d759c0c2fd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:57.851175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7ae2e-ce30-4403-bb31-d759c0c2fd8c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:57.851574 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7ae2e-ce30-4403-bb31-d759c0c2fd8c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:57.851936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7ae2e-ce30-4403-bb31-d759c0c2fd8c None admin] PATH_INFO: `/v3/users/placement` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:57.855704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:57.866633 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7ae2e-ce30-4403-bb31-d759c0c2fd8c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:57.869118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a7ae2e-ce30-4403-bb31-d759c0c2fd8c None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:57.889008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-b5a7ae2e-ce30-4403-bb31-d759c0c2fd8c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:37:57.889008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:37:57.889008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:37:57.889008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:37:57.889008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:37:57.889008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:37:57.889008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:37:57.889008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:37:57.889008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:37:57.889008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:37:57.889008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:37:57.889008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:37:57.889008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:37:57.889008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:37:57.889008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:37:57.889008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:37:57.898730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:37:57.898730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:37:57.898730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:37:57.898730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:37:57.898730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:37:57.898730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:37:57.898730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:37:57.898730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:37:57.898730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:37:57.898730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:37:57.898730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:37:57.898730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:37:57.898730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:37:57.898730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:37:57.898730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:37:57.898730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:37:57.898730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:37:57.898730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:37:57.900887 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:37:57.900887 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:37:57.900887 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:37:57.900887 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:37:57.900887 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:37:57.900887 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:37:57.900887 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:37:57.900887 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:37:57.900887 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:37:57.900887 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:37:57.900887 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 633/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:37:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:37:57.903330 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3cc1d-229b-4d12-8680-a232c21250af None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:58.053125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3cc1d-229b-4d12-8680-a232c21250af None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:58.104785 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2022afc-ba73-4cbb-9013-c9e8bdf55a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:58.105719 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2022afc-ba73-4cbb-9013-c9e8bdf55a92 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:58.106349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2022afc-ba73-4cbb-9013-c9e8bdf55a92 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:58.106920 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2022afc-ba73-4cbb-9013-c9e8bdf55a92 None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:58.109054 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2022afc-ba73-4cbb-9013-c9e8bdf55a92 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:37:58.109599 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2022afc-ba73-4cbb-9013-c9e8bdf55a92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:58.113364 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2022afc-ba73-4cbb-9013-c9e8bdf55a92 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:58.144864 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 607/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Jun 27 15:37:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:37:58.152915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7ae2e-ce30-4403-bb31-d759c0c2fd8c None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:58.324271 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7ae2e-ce30-4403-bb31-d759c0c2fd8c None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:58.388191 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae54de9-12e6-4b3b-93f2-0a5883d08936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:58.389323 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae54de9-12e6-4b3b-93f2-0a5883d08936 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:58.390044 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae54de9-12e6-4b3b-93f2-0a5883d08936 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:58.390377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae54de9-12e6-4b3b-93f2-0a5883d08936 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:58.486081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:58.496820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae54de9-12e6-4b3b-93f2-0a5883d08936 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:58.498818 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae54de9-12e6-4b3b-93f2-0a5883d08936 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:58.684407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 634/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:37:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:37:58.696196 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2022afc-ba73-4cbb-9013-c9e8bdf55a92 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:58.747861 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:58.883119 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2022afc-ba73-4cbb-9013-c9e8bdf55a92 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:58.936638 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ae731-374d-4fd0-abf3-e1010118d248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:58.937361 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ae731-374d-4fd0-abf3-e1010118d248 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:58.937664 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ae731-374d-4fd0-abf3-e1010118d248 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:58.937945 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ae731-374d-4fd0-abf3-e1010118d248 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:58.950368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ae731-374d-4fd0-abf3-e1010118d248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:58.951868 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ae731-374d-4fd0-abf3-e1010118d248 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:58.974802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-601ae731-374d-4fd0-abf3-e1010118d248 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:37:58.974802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:37:58.974802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:37:58.974802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:37:58.974802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:37:58.974802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:37:58.974802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:37:58.974802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:37:58.974802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:37:58.974802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:37:58.974802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:37:58.974802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:37:58.974802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:37:58.974802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:37:58.974802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:37:58.976092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:37:58.976092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:37:58.976092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:37:58.976092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:37:58.976092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:37:58.976092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:37:58.976092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:37:58.976092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:37:58.976092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:37:58.976092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:37:58.976092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:37:58.976092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:37:58.976092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:37:58.976092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:37:58.976092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:37:58.976092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:37:58.976092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:37:58.977144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:37:58.977144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:37:58.977144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:37:58.977144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:37:58.978864 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 608/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:37:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:37:58.985015 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae54de9-12e6-4b3b-93f2-0a5883d08936 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:59.095873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:59.103874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae54de9-12e6-4b3b-93f2-0a5883d08936 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:59.140952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f0b09-7afe-4242-87f0-678ecc561fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:59.141630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f0b09-7afe-4242-87f0-678ecc561fcf None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:59.141949 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f0b09-7afe-4242-87f0-678ecc561fcf None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:59.142230 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f0b09-7afe-4242-87f0-678ecc561fcf None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:59.142965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f0b09-7afe-4242-87f0-678ecc561fcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:37:59.143279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f0b09-7afe-4242-87f0-678ecc561fcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:59.144697 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f0b09-7afe-4242-87f0-678ecc561fcf None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:59.158443 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 635/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:37:59.164384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ae731-374d-4fd0-abf3-e1010118d248 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:59.298744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:37:59.338883 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ae731-374d-4fd0-abf3-e1010118d248 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:59.406590 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfea55e-11b8-4e7e-8cc7-99a214e7ebb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:59.408080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfea55e-11b8-4e7e-8cc7-99a214e7ebb4 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:59.408537 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfea55e-11b8-4e7e-8cc7-99a214e7ebb4 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:59.409179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfea55e-11b8-4e7e-8cc7-99a214e7ebb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:59.419781 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfea55e-11b8-4e7e-8cc7-99a214e7ebb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:59.421731 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfea55e-11b8-4e7e-8cc7-99a214e7ebb4 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:59.443222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-8cfea55e-11b8-4e7e-8cc7-99a214e7ebb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:37:59.443222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:37:59.443222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:37:59.443222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:37:59.443222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:37:59.443222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:37:59.443222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:37:59.443222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:37:59.443222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:37:59.443222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:37:59.443222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:37:59.443222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:37:59.443222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:37:59.443222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:37:59.443222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:37:59.445267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:37:59.445267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:37:59.445267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:37:59.445267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:37:59.445267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:37:59.445267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:37:59.445267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:37:59.445267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:37:59.445267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:37:59.445267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:37:59.445267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:37:59.445267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:37:59.445267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:37:59.445267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:37:59.445267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:37:59.445267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:37:59.445267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:37:59.446437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:37:59.446437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:37:59.446437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:37:59.446437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:37:59.446437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:37:59.446437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:37:59.446437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:37:59.446437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:37:59.446437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 609/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:37:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:37:59.454863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f0b09-7afe-4242-87f0-678ecc561fcf None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:59.647902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f0b09-7afe-4242-87f0-678ecc561fcf None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:59.704253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b29455-e0dd-4e5e-8f20-bb56beb21382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:59.705047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b29455-e0dd-4e5e-8f20-bb56beb21382 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:59.705514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b29455-e0dd-4e5e-8f20-bb56beb21382 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:59.705931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b29455-e0dd-4e5e-8f20-bb56beb21382 None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:59.708188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b29455-e0dd-4e5e-8f20-bb56beb21382 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:37:59.708555 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b29455-e0dd-4e5e-8f20-bb56beb21382 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:59.710883 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b29455-e0dd-4e5e-8f20-bb56beb21382 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:59.737747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 636/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:37:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:37:59.740256 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfea55e-11b8-4e7e-8cc7-99a214e7ebb4 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:37:59.875701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfea55e-11b8-4e7e-8cc7-99a214e7ebb4 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:37:59.927261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800e301-ba05-41ca-a242-edabebdb9349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:37:59.928008 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800e301-ba05-41ca-a242-edabebdb9349 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:37:59.928382 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800e301-ba05-41ca-a242-edabebdb9349 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:37:59.928758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800e301-ba05-41ca-a242-edabebdb9349 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:37:59.930282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4800e301-ba05-41ca-a242-edabebdb9349 None admin] RBAC: Adding query filter params (role.id=805851682adf4debbe99b1681c9fcdfc, scope.project.id=7be83fb6f20341139147373837320148, user.id=ffafb68f1bb941588a5217ad83ad907c) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:37:59.930859 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4800e301-ba05-41ca-a242-edabebdb9349 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:37:59.933161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4800e301-ba05-41ca-a242-edabebdb9349 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:37:59.944862 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 610/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:37:59 2022] GET /identity/v3/role_assignments?user.id=ffafb68f1bb941588a5217ad83ad907c&scope.project.id=7be83fb6f20341139147373837320148&role.id=805851682adf4debbe99b1681c9fcdfc => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:38:02.700121 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f085b3-b928-406d-a276-ed016308fbf8 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:02.701076 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f085b3-b928-406d-a276-ed016308fbf8 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:02.701076 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f085b3-b928-406d-a276-ed016308fbf8 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:02.702119 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 637/1247] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:38:02.708564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1502ab5-3ec5-4fc7-8cd6-35bc257d203c None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:02.709435 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1502ab5-3ec5-4fc7-8cd6-35bc257d203c None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:02.709435 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1502ab5-3ec5-4fc7-8cd6-35bc257d203c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:02.715038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:02.730329 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-b1502ab5-3ec5-4fc7-8cd6-35bc257d203c None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:38:02.731597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-b1502ab5-3ec5-4fc7-8cd6-35bc257d203c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:38:02.829906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 611/1248] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:38:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:38:02.855671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f085b3-b928-406d-a276-ed016308fbf8 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:02.884896 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:03.003623 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f085b3-b928-406d-a276-ed016308fbf8 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:03.045170 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a194b1-667e-4e31-841c-7116278aec8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:03.046463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a194b1-667e-4e31-841c-7116278aec8b None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:03.046828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a194b1-667e-4e31-841c-7116278aec8b None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:03.047217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a194b1-667e-4e31-841c-7116278aec8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:03.064092 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a194b1-667e-4e31-841c-7116278aec8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:38:03.076890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-a4a194b1-667e-4e31-841c-7116278aec8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:38:03.076890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:03.076890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:38:03.076890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:38:03.076890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:38:03.076890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:38:03.076890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:38:03.076890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:38:03.076890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:38:03.076890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:38:03.076890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:38:03.076890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:38:03.076890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:38:03.076890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:38:03.076890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:38:03.076890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:38:03.078972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:38:03.078972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:38:03.078972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:38:03.078972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:38:03.078972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:38:03.078972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:38:03.078972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:38:03.078972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:38:03.078972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:38:03.078972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:38:03.078972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:38:03.078972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:38:03.078972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:38:03.078972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:38:03.078972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:38:03.078972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:38:03.080414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:38:03.080414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:38:03.080414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:38:03.080414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:38:03.080414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:38:03.080414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:38:03.080414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:38:03.080414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:38:03.080414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:38:03.080414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:38:03.080414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:38:03.080414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:38:03.080414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 638/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:38:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:38:03.090196 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1502ab5-3ec5-4fc7-8cd6-35bc257d203c None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:03.242686 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1502ab5-3ec5-4fc7-8cd6-35bc257d203c None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:03.285397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:03.290460 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcb41f-1c90-4443-a614-f978163cf5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:03.291421 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcb41f-1c90-4443-a614-f978163cf5ab None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:03.292041 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcb41f-1c90-4443-a614-f978163cf5ab None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:03.292568 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcb41f-1c90-4443-a614-f978163cf5ab None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:03.293816 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcb41f-1c90-4443-a614-f978163cf5ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:03.294608 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcb41f-1c90-4443-a614-f978163cf5ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:03.296252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcb41f-1c90-4443-a614-f978163cf5ab None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:03.307485 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 612/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:38:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:38:03.314856 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a194b1-667e-4e31-841c-7116278aec8b None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:03.446871 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:03.484199 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a194b1-667e-4e31-841c-7116278aec8b None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:03.526472 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc34d0-2491-4594-a583-37ed5cadd76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:03.528210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc34d0-2491-4594-a583-37ed5cadd76a None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:03.528675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc34d0-2491-4594-a583-37ed5cadd76a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:03.529142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc34d0-2491-4594-a583-37ed5cadd76a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:03.553041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc34d0-2491-4594-a583-37ed5cadd76a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:03.554454 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc34d0-2491-4594-a583-37ed5cadd76a None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:03.575211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-a7bc34d0-2491-4594-a583-37ed5cadd76a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:38:03.575211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:03.575211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:38:03.575211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:38:03.575211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:38:03.575211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:38:03.575211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:38:03.575211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:38:03.575211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:38:03.575211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:38:03.575211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:38:03.575211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:38:03.575211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:38:03.575211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:38:03.575211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:38:03.576884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:38:03.576884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:38:03.576884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:38:03.576884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:38:03.576884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:38:03.576884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:38:03.576884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:38:03.576884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:38:03.576884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:38:03.576884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:38:03.576884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:38:03.576884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:38:03.576884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:38:03.576884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:38:03.576884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:38:03.576884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:38:03.576884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:38:03.578687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:38:03.578687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:38:03.578687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:38:03.578687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:38:03.578687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 639/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:38:03.586867 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcb41f-1c90-4443-a614-f978163cf5ab None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:03.762869 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcb41f-1c90-4443-a614-f978163cf5ab None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:03.827943 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0c879-387a-4592-b580-f84df8423e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:03.829121 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0c879-387a-4592-b580-f84df8423e68 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:03.829537 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0c879-387a-4592-b580-f84df8423e68 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:03.830095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0c879-387a-4592-b580-f84df8423e68 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:03.831454 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce0c879-387a-4592-b580-f84df8423e68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:03.832013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce0c879-387a-4592-b580-f84df8423e68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:03.835202 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce0c879-387a-4592-b580-f84df8423e68 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:03.857727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 613/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:38:03.864603 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc34d0-2491-4594-a583-37ed5cadd76a None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:04.018725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc34d0-2491-4594-a583-37ed5cadd76a None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:04.063076 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:04.078910 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17150249-4b16-4eca-a234-1830b4a6b83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:04.080235 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17150249-4b16-4eca-a234-1830b4a6b83c None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:04.080740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17150249-4b16-4eca-a234-1830b4a6b83c None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:04.081229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17150249-4b16-4eca-a234-1830b4a6b83c None admin] PATH_INFO: `/v3/users/placement` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:04.096331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17150249-4b16-4eca-a234-1830b4a6b83c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:04.098933 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17150249-4b16-4eca-a234-1830b4a6b83c None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:04.119967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-17150249-4b16-4eca-a234-1830b4a6b83c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:38:04.119967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:04.119967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:38:04.119967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:38:04.119967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:38:04.119967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:38:04.119967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:38:04.119967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:38:04.119967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:38:04.119967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:38:04.119967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:38:04.119967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:38:04.119967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:38:04.119967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:38:04.119967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:38:04.119967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:38:04.121131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:38:04.121131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:38:04.121131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:38:04.121131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:38:04.121131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:38:04.121131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:38:04.121131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:38:04.121131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:38:04.121131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:38:04.121131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:38:04.121131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:38:04.121131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:38:04.121131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:38:04.121131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:38:04.121131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:38:04.121131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:38:04.121131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:38:04.121131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:38:04.122247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:38:04.122247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:38:04.122247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:38:04.122247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:38:04.122247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:38:04.122247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:38:04.122247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:38:04.122247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:38:04.122247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:38:04.122247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:38:04.128663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 640/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:38:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:38:04.130260 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0c879-387a-4592-b580-f84df8423e68 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:04.219959 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:04.291391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0c879-387a-4592-b580-f84df8423e68 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:04.346197 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfd0aa-30bc-4781-a520-99b53cf7e376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:04.347147 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfd0aa-30bc-4781-a520-99b53cf7e376 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:04.347636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfd0aa-30bc-4781-a520-99b53cf7e376 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:04.348662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cfd0aa-30bc-4781-a520-99b53cf7e376 None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:04.351810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cfd0aa-30bc-4781-a520-99b53cf7e376 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:04.352344 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cfd0aa-30bc-4781-a520-99b53cf7e376 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:04.355149 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cfd0aa-30bc-4781-a520-99b53cf7e376 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:04.389813 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 614/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Jun 27 15:38:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:38:04.398952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17150249-4b16-4eca-a234-1830b4a6b83c None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:04.579355 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17150249-4b16-4eca-a234-1830b4a6b83c None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:04.637159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c795c49-ea41-41be-95e6-9a05a174a5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:04.638172 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c795c49-ea41-41be-95e6-9a05a174a5b1 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:04.638588 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c795c49-ea41-41be-95e6-9a05a174a5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:04.639249 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c795c49-ea41-41be-95e6-9a05a174a5b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:04.732171 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:04.749746 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c795c49-ea41-41be-95e6-9a05a174a5b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:04.751261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c795c49-ea41-41be-95e6-9a05a174a5b1 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:04.905899 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 641/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:38:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 15:38:04.912445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfd0aa-30bc-4781-a520-99b53cf7e376 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:05.075682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cfd0aa-30bc-4781-a520-99b53cf7e376 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:05.100402 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:05.132373 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb413697-32ed-4215-9459-c36ec91764d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:05.133312 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb413697-32ed-4215-9459-c36ec91764d7 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:05.133602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb413697-32ed-4215-9459-c36ec91764d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:05.133970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb413697-32ed-4215-9459-c36ec91764d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:05.150253 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb413697-32ed-4215-9459-c36ec91764d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:05.152369 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb413697-32ed-4215-9459-c36ec91764d7 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:05.180701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-eb413697-32ed-4215-9459-c36ec91764d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:38:05.180701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:05.180701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:38:05.180701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:38:05.180701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:38:05.180701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:38:05.180701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:38:05.180701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:38:05.180701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:38:05.180701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:38:05.180701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:38:05.180701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:38:05.180701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:38:05.180701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:38:05.180701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:38:05.182605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:38:05.182605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:38:05.182605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:38:05.182605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:38:05.182605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:38:05.182605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:38:05.182605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:38:05.182605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:38:05.182605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:38:05.182605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:38:05.182605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:38:05.182605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:38:05.182605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:38:05.182605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:38:05.182605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:38:05.182605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:38:05.182605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:38:05.184318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:38:05.184318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:38:05.184318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:38:05.184318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:38:05.184318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 615/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:38:05.189156 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c795c49-ea41-41be-95e6-9a05a174a5b1 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:05.373880 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:05.402794 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c795c49-ea41-41be-95e6-9a05a174a5b1 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:05.475255 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add768b-33da-4552-a883-737faacd729a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:05.476472 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add768b-33da-4552-a883-737faacd729a None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:05.477053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add768b-33da-4552-a883-737faacd729a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:05.477563 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add768b-33da-4552-a883-737faacd729a None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:05.479128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add768b-33da-4552-a883-737faacd729a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:05.479777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add768b-33da-4552-a883-737faacd729a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:05.482840 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add768b-33da-4552-a883-737faacd729a None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:05.501418 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 642/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:38:05.508380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb413697-32ed-4215-9459-c36ec91764d7 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:05.670495 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb413697-32ed-4215-9459-c36ec91764d7 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:05.723807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd9c84-43d7-4697-abd4-716b8a0ca780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:05.724887 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd9c84-43d7-4697-abd4-716b8a0ca780 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:05.725390 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd9c84-43d7-4697-abd4-716b8a0ca780 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:05.725809 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd9c84-43d7-4697-abd4-716b8a0ca780 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:05.735798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfd9c84-43d7-4697-abd4-716b8a0ca780 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:05.737779 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfd9c84-43d7-4697-abd4-716b8a0ca780 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:05.757818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-abfd9c84-43d7-4697-abd4-716b8a0ca780 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:38:05.757818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:05.757818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:38:05.757818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:38:05.757818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:38:05.757818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:38:05.757818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:38:05.757818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:38:05.757818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:38:05.757818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:38:05.757818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:38:05.757818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:38:05.757818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:38:05.757818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:38:05.757818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:38:05.759843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:38:05.759843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:38:05.759843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:38:05.759843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:38:05.759843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:38:05.759843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:38:05.759843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:38:05.759843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:38:05.759843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:38:05.759843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:38:05.759843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:38:05.759843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:38:05.759843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:38:05.759843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:38:05.759843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:38:05.759843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:38:05.759843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:38:05.761859 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:38:05.761859 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:38:05.761859 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:38:05.761859 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:38:05.761859 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:38:05.761859 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:38:05.761859 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:38:05.761859 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:38:05.761859 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 616/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:38:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:38:05.769553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add768b-33da-4552-a883-737faacd729a None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:05.907813 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add768b-33da-4552-a883-737faacd729a None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:05.955245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651938d-c10a-4fbe-99c0-21e01ac69d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:05.956079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651938d-c10a-4fbe-99c0-21e01ac69d94 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:05.956470 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651938d-c10a-4fbe-99c0-21e01ac69d94 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:05.956888 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651938d-c10a-4fbe-99c0-21e01ac69d94 None admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:05.958473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651938d-c10a-4fbe-99c0-21e01ac69d94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:05.959088 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651938d-c10a-4fbe-99c0-21e01ac69d94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:05.961005 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651938d-c10a-4fbe-99c0-21e01ac69d94 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:05.987053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 643/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:38:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:38:05.995288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd9c84-43d7-4697-abd4-716b8a0ca780 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:06.034932 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:06.163596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd9c84-43d7-4697-abd4-716b8a0ca780 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:06.219849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91985f3a-e961-4f8c-9986-0925f44c0de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:06.220996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91985f3a-e961-4f8c-9986-0925f44c0de9 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:06.221330 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91985f3a-e961-4f8c-9986-0925f44c0de9 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:06.221852 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91985f3a-e961-4f8c-9986-0925f44c0de9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:06.223731 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91985f3a-e961-4f8c-9986-0925f44c0de9 None admin] RBAC: Adding query filter params (role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.project.id=7be83fb6f20341139147373837320148, user.id=ffafb68f1bb941588a5217ad83ad907c) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:06.224157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91985f3a-e961-4f8c-9986-0925f44c0de9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:06.226625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91985f3a-e961-4f8c-9986-0925f44c0de9 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:06.235964 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 617/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:38:05 2022] GET /identity/v3/role_assignments?user.id=ffafb68f1bb941588a5217ad83ad907c&scope.project.id=7be83fb6f20341139147373837320148&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:38:08.666848 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514473f9-cbe0-4661-8171-f6e3125e7e05 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:08.667390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514473f9-cbe0-4661-8171-f6e3125e7e05 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:08.667390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514473f9-cbe0-4661-8171-f6e3125e7e05 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:08.668779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 644/1261] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:38:08.674242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a8c8e-b88e-4be2-9db1-978de09d4ab0 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:08.674774 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a8c8e-b88e-4be2-9db1-978de09d4ab0 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:08.674896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a8c8e-b88e-4be2-9db1-978de09d4ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:08.692674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-9e6a8c8e-b88e-4be2-9db1-978de09d4ab0 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:38:08.693924 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-9e6a8c8e-b88e-4be2-9db1-978de09d4ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:38:08.763997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 618/1262] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:38:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:38:08.787868 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514473f9-cbe0-4661-8171-f6e3125e7e05 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:08.799280 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:08.894170 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514473f9-cbe0-4661-8171-f6e3125e7e05 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:08.929696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3920876-5d7f-40f1-9c75-fe6111086bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:08.930434 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3920876-5d7f-40f1-9c75-fe6111086bad None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:08.930722 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3920876-5d7f-40f1-9c75-fe6111086bad None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:08.931002 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3920876-5d7f-40f1-9c75-fe6111086bad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:08.942246 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3920876-5d7f-40f1-9c75-fe6111086bad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:38:08.949923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-e3920876-5d7f-40f1-9c75-fe6111086bad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:38:08.949923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:08.949923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:38:08.949923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:38:08.949923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:38:08.949923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:38:08.949923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:38:08.949923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:38:08.949923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:38:08.949923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:38:08.949923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:38:08.949923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:38:08.949923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:38:08.949923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:38:08.949923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:38:08.949923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:38:08.950972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:38:08.950972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:38:08.950972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:38:08.950972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:38:08.950972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:38:08.950972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:38:08.950972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:38:08.950972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:38:08.950972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:38:08.950972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:38:08.950972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:38:08.950972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:38:08.950972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:38:08.950972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:38:08.950972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:38:08.950972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:38:08.951805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:38:08.951805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:38:08.951805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:38:08.951805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:38:08.951805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:38:08.951805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:38:08.951805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:38:08.951805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:38:08.951805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:38:08.951805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:38:08.951805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:38:08.951805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:38:08.951805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 645/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:38:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:38:08.958186 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a8c8e-b88e-4be2-9db1-978de09d4ab0 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:08.977751 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:09.064138 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a8c8e-b88e-4be2-9db1-978de09d4ab0 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:09.101579 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb38f7b-ee53-4f1e-be18-a27da8ce4e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:09.102295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb38f7b-ee53-4f1e-be18-a27da8ce4e92 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:09.102580 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb38f7b-ee53-4f1e-be18-a27da8ce4e92 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:09.102866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb38f7b-ee53-4f1e-be18-a27da8ce4e92 None admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:09.103640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb38f7b-ee53-4f1e-be18-a27da8ce4e92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:09.103954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb38f7b-ee53-4f1e-be18-a27da8ce4e92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:09.105342 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb38f7b-ee53-4f1e-be18-a27da8ce4e92 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:09.114884 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 619/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:38:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:38:09.120678 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3920876-5d7f-40f1-9c75-fe6111086bad None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:09.196672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:09.233179 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3920876-5d7f-40f1-9c75-fe6111086bad None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:09.268754 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75572a-abba-47c4-9b5c-7ea682d68a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:09.270110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75572a-abba-47c4-9b5c-7ea682d68a88 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:09.270397 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75572a-abba-47c4-9b5c-7ea682d68a88 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:09.270669 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75572a-abba-47c4-9b5c-7ea682d68a88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:09.283066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75572a-abba-47c4-9b5c-7ea682d68a88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:09.284803 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75572a-abba-47c4-9b5c-7ea682d68a88 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:09.308295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-5f75572a-abba-47c4-9b5c-7ea682d68a88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:38:09.308295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:09.308295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:38:09.308295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:38:09.308295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:38:09.308295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:38:09.308295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:38:09.308295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:38:09.308295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:38:09.308295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:38:09.308295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:38:09.308295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:38:09.308295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:38:09.308295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:38:09.308295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:38:09.312762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:38:09.312762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:38:09.312762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:38:09.312762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:38:09.312762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:38:09.312762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:38:09.312762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:38:09.312762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:38:09.312762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:38:09.312762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:38:09.312762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:38:09.312762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:38:09.312762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:38:09.312762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:38:09.312762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:38:09.312762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:38:09.312762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:38:09.313923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:38:09.313923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:38:09.313923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:38:09.313923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:38:09.313923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 646/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:38:09.317172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb38f7b-ee53-4f1e-be18-a27da8ce4e92 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:09.454682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:09.500013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb38f7b-ee53-4f1e-be18-a27da8ce4e92 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:09.540118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ae394-e024-4361-8c11-fff065e9ec38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:09.540797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ae394-e024-4361-8c11-fff065e9ec38 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:09.541086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ae394-e024-4361-8c11-fff065e9ec38 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:09.541365 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ae394-e024-4361-8c11-fff065e9ec38 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:09.542144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ae394-e024-4361-8c11-fff065e9ec38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:09.542454 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ae394-e024-4361-8c11-fff065e9ec38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:09.543861 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ae394-e024-4361-8c11-fff065e9ec38 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:09.558047 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 620/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:38:09.563934 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75572a-abba-47c4-9b5c-7ea682d68a88 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:09.707608 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75572a-abba-47c4-9b5c-7ea682d68a88 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:09.729818 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:09.745289 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e48392-429d-4567-bae4-3a845aea0316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:09.746143 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e48392-429d-4567-bae4-3a845aea0316 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:09.746431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e48392-429d-4567-bae4-3a845aea0316 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:09.746709 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e48392-429d-4567-bae4-3a845aea0316 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:09.755376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e48392-429d-4567-bae4-3a845aea0316 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:09.756841 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e48392-429d-4567-bae4-3a845aea0316 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:09.773210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-21e48392-429d-4567-bae4-3a845aea0316 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:38:09.773210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:09.773210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:38:09.773210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:38:09.773210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:38:09.773210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:38:09.773210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:38:09.773210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:38:09.773210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:38:09.773210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:38:09.773210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:38:09.773210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:38:09.773210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:38:09.773210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:38:09.773210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:38:09.773210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:38:09.774799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:38:09.774799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:38:09.774799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:38:09.774799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:38:09.774799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:38:09.774799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:38:09.774799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:38:09.774799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:38:09.774799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:38:09.774799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:38:09.774799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:38:09.774799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:38:09.774799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:38:09.774799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:38:09.774799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:38:09.774799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:38:09.774799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:38:09.774799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:38:09.776576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:38:09.776576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:38:09.776576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:38:09.776576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:38:09.776576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:38:09.776576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:38:09.776576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:38:09.776576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:38:09.776576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:38:09.776576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:38:09.776576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 647/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:38:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:38:09.780514 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ae394-e024-4361-8c11-fff065e9ec38 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:09.892909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ae394-e024-4361-8c11-fff065e9ec38 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:09.930489 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b723d-e88a-4021-b9b0-31da01ebbbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:09.931172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b723d-e88a-4021-b9b0-31da01ebbbe5 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:09.931452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b723d-e88a-4021-b9b0-31da01ebbbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:09.931730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b723d-e88a-4021-b9b0-31da01ebbbe5 None admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:09.932593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b723d-e88a-4021-b9b0-31da01ebbbe5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:09.932897 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b723d-e88a-4021-b9b0-31da01ebbbe5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:09.934393 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b723d-e88a-4021-b9b0-31da01ebbbe5 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:09.944797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:09.955255 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 621/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Jun 27 15:38:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:38:09.961676 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e48392-429d-4567-bae4-3a845aea0316 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:10.077891 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e48392-429d-4567-bae4-3a845aea0316 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:10.131755 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58f776-9850-4e7b-8209-c41af6c08fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:10.132584 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58f776-9850-4e7b-8209-c41af6c08fda None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:10.132874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58f776-9850-4e7b-8209-c41af6c08fda None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:10.133156 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58f776-9850-4e7b-8209-c41af6c08fda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:10.148467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58f776-9850-4e7b-8209-c41af6c08fda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:10.150006 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58f776-9850-4e7b-8209-c41af6c08fda None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:10.174013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-0d58f776-9850-4e7b-8209-c41af6c08fda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:38:10.174013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:10.174013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:38:10.174013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:38:10.174013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:38:10.174013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:38:10.174013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:38:10.174013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:38:10.174013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:38:10.174013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:38:10.174013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:38:10.174013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:38:10.174013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:38:10.174013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:38:10.174013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:38:10.175260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:38:10.175260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:38:10.175260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:38:10.175260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:38:10.175260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:38:10.175260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:38:10.175260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:38:10.175260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:38:10.175260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:38:10.175260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:38:10.175260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:38:10.175260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:38:10.175260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:38:10.175260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:38:10.175260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:38:10.175260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:38:10.175260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:38:10.176651 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:38:10.176651 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:38:10.176651 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:38:10.176651 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:38:10.176651 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 648/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:38:10.181981 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b723d-e88a-4021-b9b0-31da01ebbbe5 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:10.313157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b723d-e88a-4021-b9b0-31da01ebbbe5 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:10.352940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05095941-d653-4153-aaae-e82127759bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:10.353618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05095941-d653-4153-aaae-e82127759bdc None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:10.353934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05095941-d653-4153-aaae-e82127759bdc None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:10.354211 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05095941-d653-4153-aaae-e82127759bdc None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:10.354946 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05095941-d653-4153-aaae-e82127759bdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:10.355254 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05095941-d653-4153-aaae-e82127759bdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:10.356640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05095941-d653-4153-aaae-e82127759bdc None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:10.374283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 622/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:38:10.381482 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58f776-9850-4e7b-8209-c41af6c08fda None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:10.433199 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:10.539065 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58f776-9850-4e7b-8209-c41af6c08fda None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:10.597026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b3976-593f-4fd0-8685-bf2eb6b4c67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:10.598417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b3976-593f-4fd0-8685-bf2eb6b4c67a None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:10.598747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b3976-593f-4fd0-8685-bf2eb6b4c67a None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:10.599072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b3976-593f-4fd0-8685-bf2eb6b4c67a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:10.606911 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b3976-593f-4fd0-8685-bf2eb6b4c67a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:10.608363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b3976-593f-4fd0-8685-bf2eb6b4c67a None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:10.624217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-bb9b3976-593f-4fd0-8685-bf2eb6b4c67a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:38:10.624217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:10.624217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:38:10.624217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:38:10.624217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:38:10.624217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:38:10.624217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:38:10.624217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:38:10.624217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:38:10.624217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:38:10.624217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:38:10.624217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:38:10.624217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:38:10.624217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:38:10.624217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:38:10.625916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:38:10.625916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:38:10.625916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:38:10.625916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:38:10.625916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:38:10.625916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:38:10.625916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:38:10.625916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:38:10.625916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:38:10.625916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:38:10.625916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:38:10.625916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:38:10.625916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:38:10.625916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:38:10.625916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:38:10.625916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:38:10.625916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:38:10.626879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:38:10.626879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:38:10.626879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:38:10.626879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:38:10.626879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:38:10.626879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:38:10.626879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:38:10.626879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:38:10.626879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 649/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:38:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:38:10.631785 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05095941-d653-4153-aaae-e82127759bdc None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:10.708104 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:10.783432 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05095941-d653-4153-aaae-e82127759bdc None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:10.828418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a656d1-dc5a-4f83-819b-70f75e6d2c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:10.829194 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a656d1-dc5a-4f83-819b-70f75e6d2c63 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:10.829515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a656d1-dc5a-4f83-819b-70f75e6d2c63 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:10.829886 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a656d1-dc5a-4f83-819b-70f75e6d2c63 None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:10.830752 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a656d1-dc5a-4f83-819b-70f75e6d2c63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:10.831142 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a656d1-dc5a-4f83-819b-70f75e6d2c63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:10.833966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a656d1-dc5a-4f83-819b-70f75e6d2c63 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:10.848372 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 623/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:38:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:38:10.857023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b3976-593f-4fd0-8685-bf2eb6b4c67a None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:10.972010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:10.983087 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b3976-593f-4fd0-8685-bf2eb6b4c67a None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:11.025337 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fea90-120b-4ed3-aad6-9ae3fcc779c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:11.026276 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fea90-120b-4ed3-aad6-9ae3fcc779c3 None admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:11.026563 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fea90-120b-4ed3-aad6-9ae3fcc779c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:11.026838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fea90-120b-4ed3-aad6-9ae3fcc779c3 None admin] PATH_INFO: `/v3/projects/7be83fb6f20341139147373837320148/users/ffafb68f1bb941588a5217ad83ad907c/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:11.028942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788fea90-120b-4ed3-aad6-9ae3fcc779c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7be83fb6f20341139147373837320148, user_id=ffafb68f1bb941588a5217ad83ad907c, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:11.030419 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788fea90-120b-4ed3-aad6-9ae3fcc779c3 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:11.055597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 650/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Mon Jun 27 15:38:10 2022] PUT /identity/v3/projects/7be83fb6f20341139147373837320148/users/ffafb68f1bb941588a5217ad83ad907c/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:38:13.284194 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bc099-724f-47fe-af90-1f8bb318a1cc None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:13.284762 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bc099-724f-47fe-af90-1f8bb318a1cc None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:13.284930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bc099-724f-47fe-af90-1f8bb318a1cc None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:13.287121 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 624/1274] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:38:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:38:13.293543 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed1e4e-ccbd-4c14-8984-64ea17909934 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:13.294113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed1e4e-ccbd-4c14-8984-64ea17909934 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:13.294113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed1e4e-ccbd-4c14-8984-64ea17909934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:13.311477 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-0aed1e4e-ccbd-4c14-8984-64ea17909934 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:38:13.312713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0aed1e4e-ccbd-4c14-8984-64ea17909934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:38:13.397429 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 651/1275] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:38:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:38:13.431010 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bc099-724f-47fe-af90-1f8bb318a1cc None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:13.547812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bc099-724f-47fe-af90-1f8bb318a1cc None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:13.566133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:13.584314 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0b073-a884-4dfd-88c5-50ee58b92f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:13.585115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0b073-a884-4dfd-88c5-50ee58b92f1b None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:13.585402 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0b073-a884-4dfd-88c5-50ee58b92f1b None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:13.585702 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0b073-a884-4dfd-88c5-50ee58b92f1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:13.596659 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e0b073-a884-4dfd-88c5-50ee58b92f1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 15:38:13.604986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-19e0b073-a884-4dfd-88c5-50ee58b92f1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:38:13.604986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:13.604986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:38:13.604986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:38:13.604986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:38:13.604986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:38:13.604986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:38:13.604986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:38:13.604986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:38:13.604986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:38:13.604986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:38:13.604986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:38:13.604986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 15:38:13.604986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 15:38:13.604986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 15:38:13.604986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 15:38:13.606271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 15:38:13.606271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 15:38:13.606271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 15:38:13.606271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 15:38:13.606271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:38:13.606271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:38:13.606271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:38:13.606271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:38:13.606271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:38:13.606271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:38:13.606271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:38:13.606271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:38:13.606271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:38:13.606271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:38:13.606271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:38:13.606271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:38:13.607337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:38:13.607337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:38:13.607337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:38:13.607337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:38:13.607337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 15:38:13.607337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 15:38:13.607337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 15:38:13.607337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 15:38:13.607337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 15:38:13.607337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 15:38:13.607337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 15:38:13.607337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:38:13.607337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 625/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Mon Jun 27 15:38:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:38:13.612320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed1e4e-ccbd-4c14-8984-64ea17909934 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:13.706980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:13.728006 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed1e4e-ccbd-4c14-8984-64ea17909934 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:13.771090 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764bd005-0d5d-4cc6-9ef7-02c8aba7b5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:13.771764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bd005-0d5d-4cc6-9ef7-02c8aba7b5b9 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:13.772050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bd005-0d5d-4cc6-9ef7-02c8aba7b5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:13.772327 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bd005-0d5d-4cc6-9ef7-02c8aba7b5b9 None admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:13.773094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764bd005-0d5d-4cc6-9ef7-02c8aba7b5b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:13.773407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764bd005-0d5d-4cc6-9ef7-02c8aba7b5b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:13.774840 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764bd005-0d5d-4cc6-9ef7-02c8aba7b5b9 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:13.785189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 652/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Mon Jun 27 15:38:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:38:13.791713 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0b073-a884-4dfd-88c5-50ee58b92f1b None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:13.917578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0b073-a884-4dfd-88c5-50ee58b92f1b None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:13.977222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33201f2b-cc3d-446a-afab-f93f69937dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:13.978606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33201f2b-cc3d-446a-afab-f93f69937dff None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:13.979105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33201f2b-cc3d-446a-afab-f93f69937dff None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:13.979591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33201f2b-cc3d-446a-afab-f93f69937dff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:13.996167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33201f2b-cc3d-446a-afab-f93f69937dff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:13.997710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33201f2b-cc3d-446a-afab-f93f69937dff None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:14.019120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-33201f2b-cc3d-446a-afab-f93f69937dff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:38:14.019120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:14.019120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:38:14.019120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:38:14.019120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:38:14.019120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:38:14.019120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:38:14.019120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:38:14.019120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:38:14.019120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:38:14.019120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:38:14.019120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:38:14.019120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:38:14.019120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:38:14.019120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:38:14.021205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:38:14.021205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:38:14.021205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:38:14.021205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:38:14.021205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:38:14.021205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:38:14.021205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:38:14.021205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:38:14.021205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:38:14.021205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:38:14.021205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:38:14.021205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:38:14.021205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:38:14.021205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:38:14.021205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:38:14.021205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:38:14.021205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:38:14.022936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:38:14.022936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:38:14.022936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:38:14.022936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:38:14.022936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 626/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:38:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:38:14.028035 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764bd005-0d5d-4cc6-9ef7-02c8aba7b5b9 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:14.169819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764bd005-0d5d-4cc6-9ef7-02c8aba7b5b9 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:14.215303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82759cb5-c63b-4ba1-9a76-e18a67e56a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:14.216431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82759cb5-c63b-4ba1-9a76-e18a67e56a8d None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:14.216936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82759cb5-c63b-4ba1-9a76-e18a67e56a8d None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:14.217433 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82759cb5-c63b-4ba1-9a76-e18a67e56a8d None admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:14.219836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82759cb5-c63b-4ba1-9a76-e18a67e56a8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:14.219836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82759cb5-c63b-4ba1-9a76-e18a67e56a8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:14.221929 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82759cb5-c63b-4ba1-9a76-e18a67e56a8d None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:14.253315 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 653/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:38:14.260016 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33201f2b-cc3d-446a-afab-f93f69937dff None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:14.295942 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:14.407821 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33201f2b-cc3d-446a-afab-f93f69937dff None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:14.448772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e004121-dd02-4239-93aa-e12d6d77b95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:14.449506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e004121-dd02-4239-93aa-e12d6d77b95e None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:14.449819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e004121-dd02-4239-93aa-e12d6d77b95e None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:14.450131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e004121-dd02-4239-93aa-e12d6d77b95e None admin] PATH_INFO: `/v3/users/placement` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:14.462732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e004121-dd02-4239-93aa-e12d6d77b95e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:14.464215 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e004121-dd02-4239-93aa-e12d6d77b95e None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:14.480478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-8e004121-dd02-4239-93aa-e12d6d77b95e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:38:14.480478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:14.480478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:38:14.480478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:38:14.480478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:38:14.480478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:38:14.480478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:38:14.480478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:38:14.480478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:38:14.480478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:38:14.480478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:38:14.480478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:38:14.480478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 15:38:14.480478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 15:38:14.480478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 15:38:14.480478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 15:38:14.482470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:38:14.482470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:38:14.482470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 15:38:14.482470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:38:14.482470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 15:38:14.482470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 15:38:14.482470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:38:14.482470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:38:14.482470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:38:14.482470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:38:14.482470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:38:14.482470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:38:14.482470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:38:14.482470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:38:14.482470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:38:14.482470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:38:14.482470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:38:14.482470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:38:14.484602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:38:14.484602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:38:14.484602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 15:38:14.484602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 15:38:14.484602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 15:38:14.484602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 15:38:14.484602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 15:38:14.484602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 15:38:14.484602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 15:38:14.484602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:38:14.484602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 627/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:38:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:38:14.490389 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82759cb5-c63b-4ba1-9a76-e18a67e56a8d None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:14.494344 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:14.601488 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82759cb5-c63b-4ba1-9a76-e18a67e56a8d None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:14.648769 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429f227-be33-481a-a0fa-4ad5ad19fbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:14.649577 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429f227-be33-481a-a0fa-4ad5ad19fbb5 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:14.649987 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429f227-be33-481a-a0fa-4ad5ad19fbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:14.650336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429f227-be33-481a-a0fa-4ad5ad19fbb5 None admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:14.651330 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429f227-be33-481a-a0fa-4ad5ad19fbb5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:14.651677 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429f227-be33-481a-a0fa-4ad5ad19fbb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:14.653275 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429f227-be33-481a-a0fa-4ad5ad19fbb5 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:14.678372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 654/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Mon Jun 27 15:38:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:38:14.685428 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e004121-dd02-4239-93aa-e12d6d77b95e None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:14.794312 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:14.808512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e004121-dd02-4239-93aa-e12d6d77b95e None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:14.846276 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848961c3-49af-42a4-bd4a-25d52f3a3760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:14.846940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848961c3-49af-42a4-bd4a-25d52f3a3760 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:14.847224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848961c3-49af-42a4-bd4a-25d52f3a3760 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:14.848335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848961c3-49af-42a4-bd4a-25d52f3a3760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:14.927995 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848961c3-49af-42a4-bd4a-25d52f3a3760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:14.929460 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848961c3-49af-42a4-bd4a-25d52f3a3760 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:15.030649 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:15.043079 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 628/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Mon Jun 27 15:38:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 15:38:15.049395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429f227-be33-481a-a0fa-4ad5ad19fbb5 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:15.122004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:15.167770 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429f227-be33-481a-a0fa-4ad5ad19fbb5 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:15.208194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a63bd9-d36e-4442-901b-4554dd4569e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:15.209038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a63bd9-d36e-4442-901b-4554dd4569e6 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:15.209325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a63bd9-d36e-4442-901b-4554dd4569e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:15.209604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a63bd9-d36e-4442-901b-4554dd4569e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:15.221212 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a63bd9-d36e-4442-901b-4554dd4569e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:15.222905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a63bd9-d36e-4442-901b-4554dd4569e6 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:15.242767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-f5a63bd9-d36e-4442-901b-4554dd4569e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:38:15.242767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:15.242767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:38:15.242767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:38:15.242767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:38:15.242767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:38:15.242767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:38:15.242767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:38:15.242767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:38:15.242767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:38:15.242767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:38:15.242767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:38:15.242767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 15:38:15.242767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 15:38:15.242767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 15:38:15.244405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 15:38:15.244405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:38:15.244405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:38:15.244405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:38:15.244405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:38:15.244405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:38:15.244405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:38:15.244405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:38:15.244405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:38:15.244405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:38:15.244405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:38:15.244405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:38:15.244405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:38:15.244405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:38:15.244405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:38:15.244405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:38:15.244405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:38:15.246159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 15:38:15.246159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 15:38:15.246159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 15:38:15.246159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:38:15.246159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 655/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:38:15.251215 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848961c3-49af-42a4-bd4a-25d52f3a3760 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:15.365390 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848961c3-49af-42a4-bd4a-25d52f3a3760 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:15.406160 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e564993-fe51-4854-b7aa-325c507184a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:15.406841 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e564993-fe51-4854-b7aa-325c507184a3 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:15.407126 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e564993-fe51-4854-b7aa-325c507184a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:15.407404 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e564993-fe51-4854-b7aa-325c507184a3 None admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:15.408140 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e564993-fe51-4854-b7aa-325c507184a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:15.408450 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e564993-fe51-4854-b7aa-325c507184a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:15.409866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e564993-fe51-4854-b7aa-325c507184a3 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:15.424727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 629/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Mon Jun 27 15:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:38:15.433436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a63bd9-d36e-4442-901b-4554dd4569e6 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:15.558800 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a63bd9-d36e-4442-901b-4554dd4569e6 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:15.578066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:15.601853 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0a4c2-a242-4793-b0db-343abd294304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:15.603252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0a4c2-a242-4793-b0db-343abd294304 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:15.603656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0a4c2-a242-4793-b0db-343abd294304 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:15.603942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0a4c2-a242-4793-b0db-343abd294304 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:15.620724 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0a4c2-a242-4793-b0db-343abd294304 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:15.622228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0a4c2-a242-4793-b0db-343abd294304 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:15.637539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application [None req-7fb0a4c2-a242-4793-b0db-343abd294304 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:38:15.637539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:15.637539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:38:15.637539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:38:15.637539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:38:15.637539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:38:15.637539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:38:15.637539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:38:15.637539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:38:15.637539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:38:15.637539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:38:15.637539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:38:15.637539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 15:38:15.637539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 15:38:15.637539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 15:38:15.639421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 15:38:15.639421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:38:15.639421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:38:15.639421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:38:15.639421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:38:15.639421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:38:15.639421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:38:15.639421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:38:15.639421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application with Lock( Jun 27 15:38:15.639421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:38:15.639421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:38:15.639421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:38:15.639421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:38:15.639421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:38:15.639421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:38:15.639421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:38:15.639421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:38:15.641325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 15:38:15.641325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 15:38:15.641325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 15:38:15.641325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 15:38:15.641325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 15:38:15.641325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 15:38:15.641325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 15:38:15.641325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: ERROR keystone.server.flask.application Jun 27 15:38:15.641325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 656/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Mon Jun 27 15:38:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:38:15.648508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e564993-fe51-4854-b7aa-325c507184a3 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:15.704546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:15.761161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e564993-fe51-4854-b7aa-325c507184a3 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:15.798014 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45654364-c859-4c5e-afca-89ace238c2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:15.798805 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45654364-c859-4c5e-afca-89ace238c2b5 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:15.799181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45654364-c859-4c5e-afca-89ace238c2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:15.799602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45654364-c859-4c5e-afca-89ace238c2b5 None admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:15.800518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45654364-c859-4c5e-afca-89ace238c2b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:15.801480 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45654364-c859-4c5e-afca-89ace238c2b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:15.804193 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45654364-c859-4c5e-afca-89ace238c2b5 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:15.818158 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 630/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Mon Jun 27 15:38:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:38:15.825272 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0a4c2-a242-4793-b0db-343abd294304 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:15.963289 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0a4c2-a242-4793-b0db-343abd294304 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:16.021354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727269ae-ee4c-4a87-9556-623454340938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:16.022434 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727269ae-ee4c-4a87-9556-623454340938 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:16.022915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727269ae-ee4c-4a87-9556-623454340938 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:16.023412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727269ae-ee4c-4a87-9556-623454340938 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:16.024987 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727269ae-ee4c-4a87-9556-623454340938 None admin] RBAC: Adding query filter params (role.id=13962c72b72c40ba9bf480db87b2dcfa, scope.project.id=7be83fb6f20341139147373837320148, user.id=ffafb68f1bb941588a5217ad83ad907c) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:16.025549 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727269ae-ee4c-4a87-9556-623454340938 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:16.028305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727269ae-ee4c-4a87-9556-623454340938 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:16.040163 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 657/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Mon Jun 27 15:38:15 2022] GET /identity/v3/role_assignments?user.id=ffafb68f1bb941588a5217ad83ad907c&scope.project.id=7be83fb6f20341139147373837320148&role.id=13962c72b72c40ba9bf480db87b2dcfa => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:38:18.197269 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149654c-5723-47cd-98cc-30d9522a2b43 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:18.197905 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149654c-5723-47cd-98cc-30d9522a2b43 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:18.197905 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149654c-5723-47cd-98cc-30d9522a2b43 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:18.199248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 631/1288] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:38:18.204961 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcdb57-65c0-46c3-af3c-f2e54f856168 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:18.205487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcdb57-65c0-46c3-af3c-f2e54f856168 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:18.205609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcdb57-65c0-46c3-af3c-f2e54f856168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:18.224724 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-e6dcdb57-65c0-46c3-af3c-f2e54f856168 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:38:18.225983 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e6dcdb57-65c0-46c3-af3c-f2e54f856168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:38:18.261351 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:18.307435 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 658/1289] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:38:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:38:18.331192 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149654c-5723-47cd-98cc-30d9522a2b43 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:18.453007 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149654c-5723-47cd-98cc-30d9522a2b43 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:18.465705 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:18.492998 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f227ef1-b4ea-4dca-90b7-e125ccdd55b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:18.493848 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f227ef1-b4ea-4dca-90b7-e125ccdd55b3 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:18.494134 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f227ef1-b4ea-4dca-90b7-e125ccdd55b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:18.494410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f227ef1-b4ea-4dca-90b7-e125ccdd55b3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:18.495075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f227ef1-b4ea-4dca-90b7-e125ccdd55b3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:18.496468 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f227ef1-b4ea-4dca-90b7-e125ccdd55b3 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:18.511091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-1f227ef1-b4ea-4dca-90b7-e125ccdd55b3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 15:38:18.511091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:18.511091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:38:18.511091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:38:18.511091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:38:18.511091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:38:18.511091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:38:18.511091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:38:18.511091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 15:38:18.511091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:38:18.511091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:38:18.511091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:38:18.511091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:18.511091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:38:18.511091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:38:18.511091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:38:18.511091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:38:18.512866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:38:18.512866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:38:18.512866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:38:18.512866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:38:18.512866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:38:18.512866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:38:18.512866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:38:18.512866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:38:18.512866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:38:18.512866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:38:18.512866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:38:18.512866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:38:18.512866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:38:18.512866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:38:18.512866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:38:18.512866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:38:18.513898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:38:18.513898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:38:18.513898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:38:18.513898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:38:18.513898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:38:18.513898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:38:18.513898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:38:18.513898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:38:18.513898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:38:18.513898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:38:18.513898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:38:18.513898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:38:18.513898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 15:38:18.513898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:38:18.513898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 632/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jun 27 15:38:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:38:18.518649 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcdb57-65c0-46c3-af3c-f2e54f856168 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:18.628381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcdb57-65c0-46c3-af3c-f2e54f856168 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:18.668321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918e9a8-0a36-4107-9ef3-5b79f9cc0389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:18.668990 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918e9a8-0a36-4107-9ef3-5b79f9cc0389 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:18.669271 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918e9a8-0a36-4107-9ef3-5b79f9cc0389 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:18.669545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918e9a8-0a36-4107-9ef3-5b79f9cc0389 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:18.670289 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918e9a8-0a36-4107-9ef3-5b79f9cc0389 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:18.670595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918e9a8-0a36-4107-9ef3-5b79f9cc0389 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:18.671956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918e9a8-0a36-4107-9ef3-5b79f9cc0389 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:18.678941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 659/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Jun 27 15:38:18 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:38:18.687542 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f227ef1-b4ea-4dca-90b7-e125ccdd55b3 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:18.798164 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f227ef1-b4ea-4dca-90b7-e125ccdd55b3 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:18.838955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f35cb-fb12-4b44-b8b1-7a879b1514e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:18.839693 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f35cb-fb12-4b44-b8b1-7a879b1514e1 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:18.840049 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f35cb-fb12-4b44-b8b1-7a879b1514e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:18.840379 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f35cb-fb12-4b44-b8b1-7a879b1514e1 None admin] PATH_INFO: `/v3/services` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:18.841160 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135f35cb-fb12-4b44-b8b1-7a879b1514e1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:18.841520 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135f35cb-fb12-4b44-b8b1-7a879b1514e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:18.842989 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135f35cb-fb12-4b44-b8b1-7a879b1514e1 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:18.849075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 633/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Jun 27 15:38:18 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:38:20.989873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726977c7-352d-414d-bea1-fdfc1eab0148 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:20.990675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726977c7-352d-414d-bea1-fdfc1eab0148 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:20.990871 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726977c7-352d-414d-bea1-fdfc1eab0148 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:20.992302 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 660/1293] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:38:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:38:20.998284 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98adb4d-f124-4274-aafe-457e0f24b20d None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:20.998802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98adb4d-f124-4274-aafe-457e0f24b20d None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:20.998935 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98adb4d-f124-4274-aafe-457e0f24b20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:21.016879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-d98adb4d-f124-4274-aafe-457e0f24b20d None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:38:21.018503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d98adb4d-f124-4274-aafe-457e0f24b20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:38:21.055761 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:21.099533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 634/1294] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:38:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 15:38:21.123254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726977c7-352d-414d-bea1-fdfc1eab0148 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:21.158439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:21.241447 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726977c7-352d-414d-bea1-fdfc1eab0148 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:21.280146 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47852fe2-2370-4124-99b2-d1e4d5efb3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:21.280815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47852fe2-2370-4124-99b2-d1e4d5efb3be None admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:21.281102 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47852fe2-2370-4124-99b2-d1e4d5efb3be None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:21.281381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47852fe2-2370-4124-99b2-d1e4d5efb3be None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:21.282167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47852fe2-2370-4124-99b2-d1e4d5efb3be None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:21.283540 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47852fe2-2370-4124-99b2-d1e4d5efb3be None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:21.298659 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 661/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Mon Jun 27 15:38:21 2022] POST /identity/v3/services => generated 246 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:38:23.482432 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9f371-2ad5-43f0-a2a3-52b86c08d6c5 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:23.483103 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9f371-2ad5-43f0-a2a3-52b86c08d6c5 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:23.483103 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9f371-2ad5-43f0-a2a3-52b86c08d6c5 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:23.484393 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 635/1296] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:38:23.490489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f65bc-0e6a-45a4-9706-69207d9b9e1f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:23.490993 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f65bc-0e6a-45a4-9706-69207d9b9e1f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:23.491153 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f65bc-0e6a-45a4-9706-69207d9b9e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:23.508377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-e05f65bc-0e6a-45a4-9706-69207d9b9e1f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:38:23.509577 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e05f65bc-0e6a-45a4-9706-69207d9b9e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:38:23.593452 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 662/1297] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:38:23 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:38:23.618712 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9f371-2ad5-43f0-a2a3-52b86c08d6c5 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:23.734019 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9f371-2ad5-43f0-a2a3-52b86c08d6c5 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:23.772330 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d108f8f-f09d-4e76-a26f-a8185a8a50fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:23.773148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d108f8f-f09d-4e76-a26f-a8185a8a50fe None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:23.773436 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d108f8f-f09d-4e76-a26f-a8185a8a50fe None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:23.773747 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d108f8f-f09d-4e76-a26f-a8185a8a50fe None admin] PATH_INFO: `/v3/services/placement` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:23.774422 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d108f8f-f09d-4e76-a26f-a8185a8a50fe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:23.775835 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d108f8f-f09d-4e76-a26f-a8185a8a50fe None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:23.790154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-0d108f8f-f09d-4e76-a26f-a8185a8a50fe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 15:38:23.790154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:23.790154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:38:23.790154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:38:23.790154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:38:23.790154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:38:23.790154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:38:23.790154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:38:23.790154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 15:38:23.790154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:38:23.790154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:38:23.790154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:38:23.790154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:23.790154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:38:23.790154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:38:23.790154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:38:23.790154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:38:23.791940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:38:23.791940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:38:23.791940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:38:23.791940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:38:23.791940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:38:23.791940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:38:23.791940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:38:23.791940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:38:23.791940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:38:23.791940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:38:23.791940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:38:23.791940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:38:23.791940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:38:23.791940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:38:23.791940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:38:23.791940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:38:23.792831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:38:23.792831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:38:23.792831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:38:23.792831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:38:23.792831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:38:23.792831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:38:23.792831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:38:23.792831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:38:23.792831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:38:23.792831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:38:23.792831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:38:23.792831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:38:23.792831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 15:38:23.792831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:38:23.792831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 636/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jun 27 15:38:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 15:38:23.798465 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f65bc-0e6a-45a4-9706-69207d9b9e1f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:23.830762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:23.972316 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f65bc-0e6a-45a4-9706-69207d9b9e1f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:24.034260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063b295-2cfd-4a27-92de-d506424d2845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:24.036007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063b295-2cfd-4a27-92de-d506424d2845 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:24.036496 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063b295-2cfd-4a27-92de-d506424d2845 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:24.037024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063b295-2cfd-4a27-92de-d506424d2845 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:24.038353 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063b295-2cfd-4a27-92de-d506424d2845 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:24.038902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063b295-2cfd-4a27-92de-d506424d2845 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:24.041774 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0063b295-2cfd-4a27-92de-d506424d2845 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:24.052759 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 663/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Jun 27 15:38:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:38:24.058976 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d108f8f-f09d-4e76-a26f-a8185a8a50fe None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:24.113817 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:24.238882 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d108f8f-f09d-4e76-a26f-a8185a8a50fe None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:24.298308 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d2a5b-ba63-4c42-aa44-b2bcc8f544bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:24.299307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d2a5b-ba63-4c42-aa44-b2bcc8f544bc None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:24.299802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d2a5b-ba63-4c42-aa44-b2bcc8f544bc None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:24.300823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d2a5b-ba63-4c42-aa44-b2bcc8f544bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:24.302528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d2a5b-ba63-4c42-aa44-b2bcc8f544bc None admin] RBAC: Adding query filter params (interface=public, service_id=e54b24103beb414390030d2c4286f389, region_id=RegionOne) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:24.303091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d2a5b-ba63-4c42-aa44-b2bcc8f544bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:24.305660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d2a5b-ba63-4c42-aa44-b2bcc8f544bc None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:24.315367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 637/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Mon Jun 27 15:38:24 2022] GET /identity/v3/endpoints?service_id=e54b24103beb414390030d2c4286f389&interface=public®ion_id=RegionOne => generated 194 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:38:24.322202 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063b295-2cfd-4a27-92de-d506424d2845 None admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:24.444875 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:24.465179 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063b295-2cfd-4a27-92de-d506424d2845 None admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:24.507665 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb0aaf-47a8-4938-98ba-8a47ca620377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:24.508377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb0aaf-47a8-4938-98ba-8a47ca620377 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:24.508663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb0aaf-47a8-4938-98ba-8a47ca620377 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:24.508940 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb0aaf-47a8-4938-98ba-8a47ca620377 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:24.509682 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb0aaf-47a8-4938-98ba-8a47ca620377 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:24.511146 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb0aaf-47a8-4938-98ba-8a47ca620377 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:24.519717 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 664/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Mon Jun 27 15:38:24 2022] GET /identity/v3/services => generated 2191 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:38:26.660380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4af84-0a58-4a24-bd18-8de8e7f69943 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:26.661359 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4af84-0a58-4a24-bd18-8de8e7f69943 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:26.661359 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4af84-0a58-4a24-bd18-8de8e7f69943 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:26.662247 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 638/1302] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:38:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:38:26.669127 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc8a18-eec2-4968-b794-1e8b969f12f8 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:26.669656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc8a18-eec2-4968-b794-1e8b969f12f8 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:26.670179 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc8a18-eec2-4968-b794-1e8b969f12f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:26.699650 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-a4fc8a18-eec2-4968-b794-1e8b969f12f8 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:38:26.701313 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-a4fc8a18-eec2-4968-b794-1e8b969f12f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:38:26.810735 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 665/1303] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:38:26 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 15:38:26.854316 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4af84-0a58-4a24-bd18-8de8e7f69943 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:26.960876 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:26.968226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4af84-0a58-4a24-bd18-8de8e7f69943 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:27.006860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d1261-6f48-457e-ada0-bcc6124239b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:27.007683 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d1261-6f48-457e-ada0-bcc6124239b5 None admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:27.007969 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d1261-6f48-457e-ada0-bcc6124239b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:27.008245 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d1261-6f48-457e-ada0-bcc6124239b5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:27.008905 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d1261-6f48-457e-ada0-bcc6124239b5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:27.010319 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d1261-6f48-457e-ada0-bcc6124239b5 None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:27.023381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application [None req-d50d1261-6f48-457e-ada0-bcc6124239b5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 15:38:27.023381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:27.023381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 15:38:27.023381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 15:38:27.023381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 15:38:27.023381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 15:38:27.023381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 15:38:27.023381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:38:27.023381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 15:38:27.023381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:38:27.023381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 15:38:27.023381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:38:27.023381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 15:38:27.023381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 15:38:27.023381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 15:38:27.023381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 15:38:27.023381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 15:38:27.025430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 15:38:27.025430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 15:38:27.025430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 15:38:27.025430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 15:38:27.025430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 15:38:27.025430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 15:38:27.025430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 15:38:27.025430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 15:38:27.025430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 15:38:27.025430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 15:38:27.025430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 15:38:27.025430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 15:38:27.025430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 15:38:27.025430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 15:38:27.025430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 15:38:27.025430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 15:38:27.027557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 15:38:27.027557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application with Lock( Jun 27 15:38:27.027557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 15:38:27.027557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self._enter() Jun 27 15:38:27.027557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 15:38:27.027557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 15:38:27.027557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 15:38:27.027557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application return self.creator() Jun 27 15:38:27.027557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 15:38:27.027557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application created_value = creator( Jun 27 15:38:27.027557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 15:38:27.027557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 15:38:27.027557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 15:38:27.027557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: ERROR keystone.server.flask.application Jun 27 15:38:27.027557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 639/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Mon Jun 27 15:38:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 15:38:27.032716 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc8a18-eec2-4968-b794-1e8b969f12f8 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:27.115747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:27.140935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc8a18-eec2-4968-b794-1e8b969f12f8 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:27.182350 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cafd01-c87c-471c-9a29-cf1ca973d0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:27.183024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cafd01-c87c-471c-9a29-cf1ca973d0b5 None admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:27.183311 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cafd01-c87c-471c-9a29-cf1ca973d0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:27.183592 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cafd01-c87c-471c-9a29-cf1ca973d0b5 None admin] PATH_INFO: `/v3/services` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:27.184316 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cafd01-c87c-471c-9a29-cf1ca973d0b5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:38:27.184624 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cafd01-c87c-471c-9a29-cf1ca973d0b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:27.186019 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cafd01-c87c-471c-9a29-cf1ca973d0b5 None admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:27.192280 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 666/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Mon Jun 27 15:38:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:38:27.198491 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d1261-6f48-457e-ada0-bcc6124239b5 None admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:38:27.313674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d1261-6f48-457e-ada0-bcc6124239b5 None admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:38:27.353307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54660377-feb2-4637-a035-0c9cae5fcf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:38:27.354035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54660377-feb2-4637-a035-0c9cae5fcf2a None admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:38:27.354323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54660377-feb2-4637-a035-0c9cae5fcf2a None admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:38:27.354601 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54660377-feb2-4637-a035-0c9cae5fcf2a None admin] PATH_INFO: `/v3/endpoints` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:38:27.355398 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54660377-feb2-4637-a035-0c9cae5fcf2a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:38:27.356823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54660377-feb2-4637-a035-0c9cae5fcf2a None admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:38:27.382418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 640/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Mon Jun 27 15:38:27 2022] POST /identity/v3/endpoints => generated 331 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:38:37.189121 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=30 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:38:37.374635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:39:40.817374 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4ee3c-521c-4179-9127-6d9ee16b0242 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:40.818252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4ee3c-521c-4179-9127-6d9ee16b0242 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:40.818252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4ee3c-521c-4179-9127-6d9ee16b0242 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:40.821089 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 667/1307] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:39:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:39:40.831465 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff8a81-6dff-4eee-b357-f1390aecafca None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:40.832073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff8a81-6dff-4eee-b357-f1390aecafca None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:40.832255 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff8a81-6dff-4eee-b357-f1390aecafca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:40.983285 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-35ff8a81-6dff-4eee-b357-f1390aecafca None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:39:40.986224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-35ff8a81-6dff-4eee-b357-f1390aecafca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:41.205955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 641/1308] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:39:41.295612 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51a195-b127-4cbb-a4ea-bd05dc448d8b None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:41.296115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51a195-b127-4cbb-a4ea-bd05dc448d8b None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:41.296603 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51a195-b127-4cbb-a4ea-bd05dc448d8b None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:41.299850 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 668/1309] 10.222.0.20 () {54 vars in 964 bytes} [Mon Jun 27 15:39:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:39:41.310331 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632d098-064c-458f-8343-389bfc2b3ecc None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:41.310836 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632d098-064c-458f-8343-389bfc2b3ecc None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:41.311327 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632d098-064c-458f-8343-389bfc2b3ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:41.472592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-f632d098-064c-458f-8343-389bfc2b3ecc None None] MFA Rules not processed for user `42dee639e2834d99b7468f911c37cf8f`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:39:41.474814 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f632d098-064c-458f-8343-389bfc2b3ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:41.518400 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:39:41.643611 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 642/1310] 10.222.0.20 () {58 vars in 1061 bytes} [Mon Jun 27 15:39:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:39:41.660375 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51a195-b127-4cbb-a4ea-bd05dc448d8b None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:39:41.663659 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-3e51a195-b127-4cbb-a4ea-bd05dc448d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:41.868048 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51a195-b127-4cbb-a4ea-bd05dc448d8b None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:39:42.019190 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59400357-d0a3-49dc-bfc1-3eecac622109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:39:42.019190 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59400357-d0a3-49dc-bfc1-3eecac622109 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:42.019531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59400357-d0a3-49dc-bfc1-3eecac622109 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:42.019877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59400357-d0a3-49dc-bfc1-3eecac622109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:42.023329 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-59400357-d0a3-49dc-bfc1-3eecac622109 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:42.135145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:39:42.213155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59400357-d0a3-49dc-bfc1-3eecac622109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:39:42.215006 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59400357-d0a3-49dc-bfc1-3eecac622109 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:39:42.240896 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 669/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:39:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:39:45.545037 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cbe54-129b-4cb0-bdbf-ce3a86a8504c None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:45.545613 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cbe54-129b-4cb0-bdbf-ce3a86a8504c None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:45.546007 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cbe54-129b-4cb0-bdbf-ce3a86a8504c None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:45.549134 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 643/1312] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:39:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:39:45.556922 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466d4b7-0dbc-4d00-83cc-7a5581e2141b None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:45.557463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466d4b7-0dbc-4d00-83cc-7a5581e2141b None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:45.557569 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466d4b7-0dbc-4d00-83cc-7a5581e2141b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:45.659056 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-4466d4b7-0dbc-4d00-83cc-7a5581e2141b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:39:45.660271 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-4466d4b7-0dbc-4d00-83cc-7a5581e2141b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:45.792499 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 670/1313] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:39:45.855333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17b2c9-7738-4084-925c-a4793a8afdba None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:45.855850 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17b2c9-7738-4084-925c-a4793a8afdba None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:45.856356 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17b2c9-7738-4084-925c-a4793a8afdba None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:45.859120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 644/1314] 10.222.0.20 () {54 vars in 964 bytes} [Mon Jun 27 15:39:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:39:45.866475 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39214f-0a9a-46b5-8161-c932fd064878 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:45.866894 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39214f-0a9a-46b5-8161-c932fd064878 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:45.867261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39214f-0a9a-46b5-8161-c932fd064878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:45.880260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:39:45.958864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-2b39214f-0a9a-46b5-8161-c932fd064878 None None] MFA Rules not processed for user `42dee639e2834d99b7468f911c37cf8f`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:39:45.960096 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2b39214f-0a9a-46b5-8161-c932fd064878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:46.071879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 671/1315] 10.222.0.20 () {58 vars in 1061 bytes} [Mon Jun 27 15:39:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:39:46.084482 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17b2c9-7738-4084-925c-a4793a8afdba None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:39:46.086103 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-ca17b2c9-7738-4084-925c-a4793a8afdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:46.215941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17b2c9-7738-4084-925c-a4793a8afdba None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:39:46.310440 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:39:46.316254 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09ceac-03cc-4844-a5a7-b4806088d737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:39:46.317546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09ceac-03cc-4844-a5a7-b4806088d737 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:46.318366 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09ceac-03cc-4844-a5a7-b4806088d737 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:46.318848 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09ceac-03cc-4844-a5a7-b4806088d737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:46.322257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-ae09ceac-03cc-4844-a5a7-b4806088d737 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:46.467920 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09ceac-03cc-4844-a5a7-b4806088d737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:39:46.469377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09ceac-03cc-4844-a5a7-b4806088d737 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:39:46.490368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 645/1316] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:39:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:39:49.176453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6596726-8d24-4139-a579-61161f7ab244 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:49.177016 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6596726-8d24-4139-a579-61161f7ab244 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:49.177016 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6596726-8d24-4139-a579-61161f7ab244 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:49.178503 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 672/1317] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:39:49.184359 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1db48-0f1b-4845-a383-faf8a2d55ee0 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:49.184848 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1db48-0f1b-4845-a383-faf8a2d55ee0 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:49.184965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1db48-0f1b-4845-a383-faf8a2d55ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:49.261764 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-dac1db48-0f1b-4845-a383-faf8a2d55ee0 None None] MFA Rules not processed for user `5c685740a045443997f807d9439f89c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:39:49.263027 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-dac1db48-0f1b-4845-a383-faf8a2d55ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:49.329991 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:39:49.416502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 646/1318] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:39:49 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:39:49.458341 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6596726-8d24-4139-a579-61161f7ab244 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:39:49.459941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-d6596726-8d24-4139-a579-61161f7ab244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:49.579388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6596726-8d24-4139-a579-61161f7ab244 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:39:49.582777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:39:49.660492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca24b03-209e-46b1-a27a-49919e8c93f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:39:49.661143 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca24b03-209e-46b1-a27a-49919e8c93f1 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:49.661430 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca24b03-209e-46b1-a27a-49919e8c93f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:49.661749 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca24b03-209e-46b1-a27a-49919e8c93f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:49.663863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-7ca24b03-209e-46b1-a27a-49919e8c93f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:49.816522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca24b03-209e-46b1-a27a-49919e8c93f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:39:49.819338 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca24b03-209e-46b1-a27a-49919e8c93f1 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:39:49.857677 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 673/1319] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:39:49 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:39:52.777657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57db5c4-de9a-400c-8741-0531e8047f31 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:52.778259 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57db5c4-de9a-400c-8741-0531e8047f31 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:52.778259 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57db5c4-de9a-400c-8741-0531e8047f31 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:52.779670 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 647/1320] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:39:52.786661 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3e97e-1eaf-4de6-b77e-b73f3dbe055c None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:52.787178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3e97e-1eaf-4de6-b77e-b73f3dbe055c None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:52.787293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3e97e-1eaf-4de6-b77e-b73f3dbe055c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:52.845153 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:39:52.882157 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-47b3e97e-1eaf-4de6-b77e-b73f3dbe055c None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:39:52.883434 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-47b3e97e-1eaf-4de6-b77e-b73f3dbe055c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:52.987665 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 674/1321] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:39:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:39:53.027169 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57db5c4-de9a-400c-8741-0531e8047f31 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:39:53.028779 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-b57db5c4-de9a-400c-8741-0531e8047f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:53.147106 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57db5c4-de9a-400c-8741-0531e8047f31 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:39:53.225925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b75b1-410b-444d-91d9-d7216090a2d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:39:53.226581 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b75b1-410b-444d-91d9-d7216090a2d3 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:53.226863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b75b1-410b-444d-91d9-d7216090a2d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:53.227135 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b75b1-410b-444d-91d9-d7216090a2d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:53.228809 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-079b75b1-410b-444d-91d9-d7216090a2d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:53.237119 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:39:53.354456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b75b1-410b-444d-91d9-d7216090a2d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:39:53.355923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b75b1-410b-444d-91d9-d7216090a2d3 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:39:53.375609 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 648/1322] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:39:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:39:56.395052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4a47b-f267-45ac-ac08-3f8c31fabf63 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:56.395670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4a47b-f267-45ac-ac08-3f8c31fabf63 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:56.395670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4a47b-f267-45ac-ac08-3f8c31fabf63 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:56.397005 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 675/1323] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:39:56.402795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a71266-18cf-4cee-86a1-e0a6832f8401 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:56.403593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a71266-18cf-4cee-86a1-e0a6832f8401 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:56.403593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a71266-18cf-4cee-86a1-e0a6832f8401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:56.483163 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-39a71266-18cf-4cee-86a1-e0a6832f8401 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:39:56.485174 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-39a71266-18cf-4cee-86a1-e0a6832f8401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:56.556399 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:39:56.603339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 649/1324] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:39:56.644187 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4a47b-f267-45ac-ac08-3f8c31fabf63 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:39:56.645963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-12b4a47b-f267-45ac-ac08-3f8c31fabf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:56.780880 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4a47b-f267-45ac-ac08-3f8c31fabf63 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:39:56.797880 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:39:56.886108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e47dd8-ca0f-4d9f-ba95-43a980b3ccd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:39:56.886795 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e47dd8-ca0f-4d9f-ba95-43a980b3ccd0 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:56.887080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e47dd8-ca0f-4d9f-ba95-43a980b3ccd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:56.887357 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e47dd8-ca0f-4d9f-ba95-43a980b3ccd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:56.889183 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-95e47dd8-ca0f-4d9f-ba95-43a980b3ccd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:57.028796 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e47dd8-ca0f-4d9f-ba95-43a980b3ccd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:39:57.030232 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e47dd8-ca0f-4d9f-ba95-43a980b3ccd0 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:39:57.052817 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 676/1325] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:39:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:39:59.548963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624c862-302d-4ceb-bd88-78e10ba46cea None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:59.549900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624c862-302d-4ceb-bd88-78e10ba46cea None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:59.549900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624c862-302d-4ceb-bd88-78e10ba46cea None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:59.550824 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 650/1326] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:39:59.556422 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad0e38-a935-44b3-aca9-2ecce0f25249 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:39:59.557101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad0e38-a935-44b3-aca9-2ecce0f25249 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:39:59.557101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad0e38-a935-44b3-aca9-2ecce0f25249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:39:59.610978 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:39:59.633502 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-30ad0e38-a935-44b3-aca9-2ecce0f25249 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:39:59.634765 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-30ad0e38-a935-44b3-aca9-2ecce0f25249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:59.751505 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 677/1327] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:39:59.789453 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624c862-302d-4ceb-bd88-78e10ba46cea None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:39:59.791091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-8624c862-302d-4ceb-bd88-78e10ba46cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:39:59.916723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624c862-302d-4ceb-bd88-78e10ba46cea None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:00.006730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49d617-b4ee-4ef3-ba9a-989aea4e8ff3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:00.007391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49d617-b4ee-4ef3-ba9a-989aea4e8ff3 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:00.007678 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49d617-b4ee-4ef3-ba9a-989aea4e8ff3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:00.007954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49d617-b4ee-4ef3-ba9a-989aea4e8ff3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:00.009673 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4e49d617-b4ee-4ef3-ba9a-989aea4e8ff3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:00.027576 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:40:00.131953 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49d617-b4ee-4ef3-ba9a-989aea4e8ff3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:00.134571 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49d617-b4ee-4ef3-ba9a-989aea4e8ff3 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:00.161764 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 651/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:39:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:40:02.479927 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c0c98-91f7-4b8e-bb7d-adf772f6f34b None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:02.480460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c0c98-91f7-4b8e-bb7d-adf772f6f34b None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:02.480460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c0c98-91f7-4b8e-bb7d-adf772f6f34b None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:02.482000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 678/1329] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:02.488121 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5f24b-e1db-40a5-8b03-7100c0fbe968 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:02.488638 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5f24b-e1db-40a5-8b03-7100c0fbe968 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:02.488946 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5f24b-e1db-40a5-8b03-7100c0fbe968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:02.582352 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-5af5f24b-e1db-40a5-8b03-7100c0fbe968 None None] MFA Rules not processed for user `5c685740a045443997f807d9439f89c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:02.583709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-5af5f24b-e1db-40a5-8b03-7100c0fbe968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:02.684914 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:40:02.723034 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 652/1330] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:40:02 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:02.762324 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c0c98-91f7-4b8e-bb7d-adf772f6f34b None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:40:02.763900 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f09c0c98-91f7-4b8e-bb7d-adf772f6f34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:02.900250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c0c98-91f7-4b8e-bb7d-adf772f6f34b None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:02.935348 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:40:03.007003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd8439-0583-4dcb-8d9a-2ec1f196c112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:03.008154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd8439-0583-4dcb-8d9a-2ec1f196c112 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:03.008440 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd8439-0583-4dcb-8d9a-2ec1f196c112 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:03.008723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd8439-0583-4dcb-8d9a-2ec1f196c112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:03.010567 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-95cd8439-0583-4dcb-8d9a-2ec1f196c112 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:03.176457 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd8439-0583-4dcb-8d9a-2ec1f196c112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:03.177982 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd8439-0583-4dcb-8d9a-2ec1f196c112 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:03.205952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 679/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:40:02 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:40:05.613343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7164e-fbe8-4a11-9650-a186aef72b62 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:05.614008 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7164e-fbe8-4a11-9650-a186aef72b62 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:05.614008 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7164e-fbe8-4a11-9650-a186aef72b62 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:05.615328 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 653/1332] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:40:05.621185 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be86f94-1b69-4567-a601-7f772b8e5997 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:05.621838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be86f94-1b69-4567-a601-7f772b8e5997 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:05.621838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be86f94-1b69-4567-a601-7f772b8e5997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:05.693576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:40:05.696965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-8be86f94-1b69-4567-a601-7f772b8e5997 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:05.698851 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8be86f94-1b69-4567-a601-7f772b8e5997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:05.806634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 680/1333] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:40:05.846387 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7164e-fbe8-4a11-9650-a186aef72b62 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:40:05.847983 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-77e7164e-fbe8-4a11-9650-a186aef72b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:05.967137 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7164e-fbe8-4a11-9650-a186aef72b62 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:06.083380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13de69-d709-4dac-ab46-6778911159f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:06.084210 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13de69-d709-4dac-ab46-6778911159f7 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:06.084334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13de69-d709-4dac-ab46-6778911159f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:06.084612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13de69-d709-4dac-ab46-6778911159f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:06.086505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3e13de69-d709-4dac-ab46-6778911159f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:06.121393 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:40:06.250405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13de69-d709-4dac-ab46-6778911159f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:06.251864 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13de69-d709-4dac-ab46-6778911159f7 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:06.271541 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 654/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:40:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:40:08.630455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e0526-78c4-47e1-843f-8fa456f3508e None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:08.631289 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e0526-78c4-47e1-843f-8fa456f3508e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:08.631473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e0526-78c4-47e1-843f-8fa456f3508e None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:08.633135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 681/1335] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:40:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:08.638834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bd594-d2f2-40df-aab2-70856fc7b700 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:08.639349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bd594-d2f2-40df-aab2-70856fc7b700 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:08.639460 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bd594-d2f2-40df-aab2-70856fc7b700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:08.709326 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-fd4bd594-d2f2-40df-aab2-70856fc7b700 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:08.710617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-fd4bd594-d2f2-40df-aab2-70856fc7b700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:08.790273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:40:08.816091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 655/1336] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:08.858561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e0526-78c4-47e1-843f-8fa456f3508e None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:40:08.860352 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-485e0526-78c4-47e1-843f-8fa456f3508e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:08.977042 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e0526-78c4-47e1-843f-8fa456f3508e None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:09.010914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:40:09.060524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf0ed4-9f5f-48b8-9dca-576c97df2ca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:09.061184 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf0ed4-9f5f-48b8-9dca-576c97df2ca9 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:09.061467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf0ed4-9f5f-48b8-9dca-576c97df2ca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:09.061777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf0ed4-9f5f-48b8-9dca-576c97df2ca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:09.063994 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-98bf0ed4-9f5f-48b8-9dca-576c97df2ca9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:09.199154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf0ed4-9f5f-48b8-9dca-576c97df2ca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:09.200635 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf0ed4-9f5f-48b8-9dca-576c97df2ca9 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:09.223745 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 682/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:40:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:40:11.701178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b251b8-1ba9-43b6-a91f-3859d6e23319 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:11.701760 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b251b8-1ba9-43b6-a91f-3859d6e23319 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:11.701760 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b251b8-1ba9-43b6-a91f-3859d6e23319 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:11.704523 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 656/1338] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:40:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:11.712485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad15527-e110-4d20-92ec-46ca7a06a272 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:11.713061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad15527-e110-4d20-92ec-46ca7a06a272 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:11.713061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad15527-e110-4d20-92ec-46ca7a06a272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:11.792843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-3ad15527-e110-4d20-92ec-46ca7a06a272 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:11.794090 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-3ad15527-e110-4d20-92ec-46ca7a06a272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:11.805387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:40:11.910211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 683/1339] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:40:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:40:11.947906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b251b8-1ba9-43b6-a91f-3859d6e23319 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:40:11.950563 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-90b251b8-1ba9-43b6-a91f-3859d6e23319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:12.071530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b251b8-1ba9-43b6-a91f-3859d6e23319 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:12.075183 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec4ae2-193b-4e80-b26b-ec692ee8c260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:12.075838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec4ae2-193b-4e80-b26b-ec692ee8c260 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:12.076131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec4ae2-193b-4e80-b26b-ec692ee8c260 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:12.076410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec4ae2-193b-4e80-b26b-ec692ee8c260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:12.089102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec4ae2-193b-4e80-b26b-ec692ee8c260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:12.090619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec4ae2-193b-4e80-b26b-ec692ee8c260 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:12.103508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 657/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:40:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:40:16.288824 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908db53-171c-4820-8eaa-df4906715f1f None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:16.289789 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908db53-171c-4820-8eaa-df4906715f1f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:16.289789 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908db53-171c-4820-8eaa-df4906715f1f None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:16.290924 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 684/1341] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:16.296441 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903c74b-3db5-4ce7-b8eb-90a0c8fc4768 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:16.297268 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903c74b-3db5-4ce7-b8eb-90a0c8fc4768 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:16.297560 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903c74b-3db5-4ce7-b8eb-90a0c8fc4768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:16.329367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-4903c74b-3db5-4ce7-b8eb-90a0c8fc4768 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:16.330666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4903c74b-3db5-4ce7-b8eb-90a0c8fc4768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:16.416109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 658/1342] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:40:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:16.455683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908db53-171c-4820-8eaa-df4906715f1f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:40:16.466313 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908db53-171c-4820-8eaa-df4906715f1f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:16.470618 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c2460-2bdd-45c3-a2d3-afcb8537dba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:16.471267 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c2460-2bdd-45c3-a2d3-afcb8537dba3 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:16.471553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c2460-2bdd-45c3-a2d3-afcb8537dba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:16.471832 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c2460-2bdd-45c3-a2d3-afcb8537dba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:16.481122 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c2460-2bdd-45c3-a2d3-afcb8537dba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:16.482921 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c2460-2bdd-45c3-a2d3-afcb8537dba3 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:16.493466 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 685/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:40:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:40:19.146280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607e474-42e3-4836-81fd-d060dc1dfd16 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:19.146837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607e474-42e3-4836-81fd-d060dc1dfd16 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:19.146837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607e474-42e3-4836-81fd-d060dc1dfd16 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:19.148229 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 659/1344] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:19.153427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a267811-bb9f-4b35-8d4d-60c0b93bf611 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:19.153911 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a267811-bb9f-4b35-8d4d-60c0b93bf611 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:19.154040 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a267811-bb9f-4b35-8d4d-60c0b93bf611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:19.173539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-5a267811-bb9f-4b35-8d4d-60c0b93bf611 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:19.174772 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-5a267811-bb9f-4b35-8d4d-60c0b93bf611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:19.182755 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 686/1345] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:40:19 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:19.226270 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607e474-42e3-4836-81fd-d060dc1dfd16 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:40:19.237711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607e474-42e3-4836-81fd-d060dc1dfd16 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:19.241192 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b87e23-7093-4886-bf3a-f5cfd41490c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:19.241861 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b87e23-7093-4886-bf3a-f5cfd41490c3 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:19.242155 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b87e23-7093-4886-bf3a-f5cfd41490c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:19.242422 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b87e23-7093-4886-bf3a-f5cfd41490c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:19.252674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b87e23-7093-4886-bf3a-f5cfd41490c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:19.254139 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b87e23-7093-4886-bf3a-f5cfd41490c3 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:19.263834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 660/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:40:19 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:40:23.412714 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8ba83-9863-4fb3-80ff-bc154f32dd99 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:23.413489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8ba83-9863-4fb3-80ff-bc154f32dd99 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:23.413489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8ba83-9863-4fb3-80ff-bc154f32dd99 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:23.414926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 687/1347] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:23.420377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785d7bd-89e6-4ffb-9dcc-6e2b2c09b4dd None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:23.422041 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785d7bd-89e6-4ffb-9dcc-6e2b2c09b4dd None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:23.422041 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785d7bd-89e6-4ffb-9dcc-6e2b2c09b4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:23.440432 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-1785d7bd-89e6-4ffb-9dcc-6e2b2c09b4dd None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:23.441662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1785d7bd-89e6-4ffb-9dcc-6e2b2c09b4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:23.450445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 661/1348] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:40:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:40:23.491126 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8ba83-9863-4fb3-80ff-bc154f32dd99 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:40:23.503075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8ba83-9863-4fb3-80ff-bc154f32dd99 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:23.506642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6fb81-4dbc-4372-8e39-60b886c6f6a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:23.507388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6fb81-4dbc-4372-8e39-60b886c6f6a4 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:23.507731 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6fb81-4dbc-4372-8e39-60b886c6f6a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:23.508073 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6fb81-4dbc-4372-8e39-60b886c6f6a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:23.518009 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6fb81-4dbc-4372-8e39-60b886c6f6a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:23.519329 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c6fb81-4dbc-4372-8e39-60b886c6f6a4 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:23.529137 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 688/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:40:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:40:28.717139 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4442b6-7479-48e4-a2d3-5f34201a6d3a None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:28.718321 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4442b6-7479-48e4-a2d3-5f34201a6d3a None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:28.718321 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4442b6-7479-48e4-a2d3-5f34201a6d3a None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:28.720198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 662/1350] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jun 27 15:40:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:28.725934 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef1249-2f4e-49b9-b937-79854665eb31 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:28.726375 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef1249-2f4e-49b9-b937-79854665eb31 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:28.726782 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef1249-2f4e-49b9-b937-79854665eb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:28.834631 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-56ef1249-2f4e-49b9-b937-79854665eb31 None None] MFA Rules not processed for user `ffafb68f1bb941588a5217ad83ad907c`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:28.835829 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-56ef1249-2f4e-49b9-b937-79854665eb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:28.921538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:40:28.959079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 689/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jun 27 15:40:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:33.434449 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=98 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:40:33.538166 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480cd87-ed91-4183-ae14-d5668f3fb727 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:33.538698 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480cd87-ed91-4183-ae14-d5668f3fb727 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:33.539437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480cd87-ed91-4183-ae14-d5668f3fb727 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:33.540984 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 663/1352] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jun 27 15:40:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:33.546788 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929547e-a174-484a-98d4-c0526e1e09fc None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:33.547540 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929547e-a174-484a-98d4-c0526e1e09fc None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:33.547540 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929547e-a174-484a-98d4-c0526e1e09fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:33.580236 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-5929547e-a174-484a-98d4-c0526e1e09fc None None] MFA Rules not processed for user `ffafb68f1bb941588a5217ad83ad907c`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:33.583094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-5929547e-a174-484a-98d4-c0526e1e09fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:33.606877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 690/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jun 27 15:40:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:35.542174 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86d539-9fa4-4e14-af18-21e6b92208f6 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:35.542669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86d539-9fa4-4e14-af18-21e6b92208f6 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:35.542669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86d539-9fa4-4e14-af18-21e6b92208f6 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:35.544057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 664/1354] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jun 27 15:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:35.549810 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb40c4-ccb4-4ce4-bddb-0fa4d6690285 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:35.550287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb40c4-ccb4-4ce4-bddb-0fa4d6690285 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:35.550485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb40c4-ccb4-4ce4-bddb-0fa4d6690285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:35.569608 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-23fb40c4-ccb4-4ce4-bddb-0fa4d6690285 None None] MFA Rules not processed for user `ffafb68f1bb941588a5217ad83ad907c`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:35.570956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-23fb40c4-ccb4-4ce4-bddb-0fa4d6690285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:35.580191 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 691/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jun 27 15:40:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:39.208764 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529792d-4e3a-4f72-9ec4-00ce59c8b32c None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:39.209311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529792d-4e3a-4f72-9ec4-00ce59c8b32c None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:39.209706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529792d-4e3a-4f72-9ec4-00ce59c8b32c None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:39.211359 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 665/1356] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:39.218505 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02fd77-7018-44cd-aece-c0d112971b0b None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:39.220327 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02fd77-7018-44cd-aece-c0d112971b0b None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:39.220969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02fd77-7018-44cd-aece-c0d112971b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:39.247957 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-fe02fd77-7018-44cd-aece-c0d112971b0b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:39.249337 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-fe02fd77-7018-44cd-aece-c0d112971b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:39.260783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 692/1357] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:39.323390 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62449c51-0467-4685-8372-0f1bde6c4d6f None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:39.323678 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62449c51-0467-4685-8372-0f1bde6c4d6f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:39.323954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62449c51-0467-4685-8372-0f1bde6c4d6f None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:39.325333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 666/1358] 10.222.0.20 () {54 vars in 961 bytes} [Mon Jun 27 15:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:39.335252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32f49b-932c-4d00-9cd8-d27c7b3e2e8b None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:39.335686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32f49b-932c-4d00-9cd8-d27c7b3e2e8b None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:39.336064 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32f49b-932c-4d00-9cd8-d27c7b3e2e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:39.410744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-fe32f49b-932c-4d00-9cd8-d27c7b3e2e8b None None] MFA Rules not processed for user `1c8f6262cf24419daf803ae71aed24f5`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:39.412171 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-fe32f49b-932c-4d00-9cd8-d27c7b3e2e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:39.513725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 693/1359] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Jun 27 15:40:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:39.520560 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62449c51-0467-4685-8372-0f1bde6c4d6f None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:40:39.537229 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62449c51-0467-4685-8372-0f1bde6c4d6f None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:39.543588 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c53adb-0070-4095-9ab3-91422860b2a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:39.544498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c53adb-0070-4095-9ab3-91422860b2a2 service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:39.544942 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c53adb-0070-4095-9ab3-91422860b2a2 service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:39.545371 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c53adb-0070-4095-9ab3-91422860b2a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:39.555556 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c53adb-0070-4095-9ab3-91422860b2a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:39.557388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c53adb-0070-4095-9ab3-91422860b2a2 service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:39.570105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 667/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:40:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:40:41.976487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c5f36-9fcc-4213-acea-b580a8bd08a4 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:41.977014 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c5f36-9fcc-4213-acea-b580a8bd08a4 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:41.977014 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c5f36-9fcc-4213-acea-b580a8bd08a4 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:41.979032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 694/1361] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:41.984527 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b617ebd-5586-431d-b7da-5f6ad642bea9 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:41.985492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b617ebd-5586-431d-b7da-5f6ad642bea9 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:41.986322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b617ebd-5586-431d-b7da-5f6ad642bea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:42.007286 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-8b617ebd-5586-431d-b7da-5f6ad642bea9 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:42.009504 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-8b617ebd-5586-431d-b7da-5f6ad642bea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:42.018862 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 668/1362] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:42.064693 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c5f36-9fcc-4213-acea-b580a8bd08a4 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:40:42.083014 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c5f36-9fcc-4213-acea-b580a8bd08a4 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:42.088938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e2443-54af-4f4b-a4bd-655040ded219 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:42.089656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e2443-54af-4f4b-a4bd-655040ded219 service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:42.090063 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e2443-54af-4f4b-a4bd-655040ded219 service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:42.090440 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e2443-54af-4f4b-a4bd-655040ded219 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:42.100520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e2443-54af-4f4b-a4bd-655040ded219 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:42.102360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e2443-54af-4f4b-a4bd-655040ded219 service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:42.121878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 695/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:40:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:40:44.602927 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3439f4-a592-4f6c-b809-0d7e341c9767 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:44.603492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3439f4-a592-4f6c-b809-0d7e341c9767 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:44.603492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3439f4-a592-4f6c-b809-0d7e341c9767 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:44.604856 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 669/1364] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:44.614817 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224457c7-9dda-4c33-8feb-2718de502ab9 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:44.615342 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224457c7-9dda-4c33-8feb-2718de502ab9 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:44.615437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224457c7-9dda-4c33-8feb-2718de502ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:44.635242 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-224457c7-9dda-4c33-8feb-2718de502ab9 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:44.636463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-224457c7-9dda-4c33-8feb-2718de502ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:44.645910 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 696/1365] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:40:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:44.692933 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919295b-46a9-456c-a96a-7b42a6816882 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:44.693278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919295b-46a9-456c-a96a-7b42a6816882 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:44.693695 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919295b-46a9-456c-a96a-7b42a6816882 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:44.695147 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 670/1366] 10.222.0.20 () {54 vars in 961 bytes} [Mon Jun 27 15:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:44.700757 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d81636-91a2-47c4-b2e0-158cabc87349 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:44.701048 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d81636-91a2-47c4-b2e0-158cabc87349 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:44.701351 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d81636-91a2-47c4-b2e0-158cabc87349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:44.720581 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-58d81636-91a2-47c4-b2e0-158cabc87349 None None] MFA Rules not processed for user `1c8f6262cf24419daf803ae71aed24f5`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:44.721808 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-58d81636-91a2-47c4-b2e0-158cabc87349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:44.730923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 697/1367] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Jun 27 15:40:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:40:44.738700 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919295b-46a9-456c-a96a-7b42a6816882 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:40:44.753337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919295b-46a9-456c-a96a-7b42a6816882 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:44.758944 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba134a71-7f97-495e-8c36-bcd6655b5165 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:44.759591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba134a71-7f97-495e-8c36-bcd6655b5165 service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:44.759875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba134a71-7f97-495e-8c36-bcd6655b5165 service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:44.760155 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba134a71-7f97-495e-8c36-bcd6655b5165 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:44.769258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba134a71-7f97-495e-8c36-bcd6655b5165 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:44.770876 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba134a71-7f97-495e-8c36-bcd6655b5165 service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:44.781986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 671/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:40:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:40:46.917896 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d713f-87f4-41d4-977a-fc154f151e5f None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:46.918778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d713f-87f4-41d4-977a-fc154f151e5f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:46.919059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d713f-87f4-41d4-977a-fc154f151e5f None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:46.920515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 698/1369] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:40:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:46.927118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c410616-1079-4500-bd76-1f72eb997a65 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:46.927644 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c410616-1079-4500-bd76-1f72eb997a65 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:46.927644 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c410616-1079-4500-bd76-1f72eb997a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:46.947981 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-7c410616-1079-4500-bd76-1f72eb997a65 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:46.949906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-7c410616-1079-4500-bd76-1f72eb997a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:46.960921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 672/1370] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:47.006987 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d713f-87f4-41d4-977a-fc154f151e5f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:40:47.022447 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d713f-87f4-41d4-977a-fc154f151e5f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:47.027920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb5b604-3060-4502-8663-5d85d7490ef3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:47.028556 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5b604-3060-4502-8663-5d85d7490ef3 service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:47.029003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5b604-3060-4502-8663-5d85d7490ef3 service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:47.029430 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5b604-3060-4502-8663-5d85d7490ef3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:47.038373 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb5b604-3060-4502-8663-5d85d7490ef3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:47.040214 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb5b604-3060-4502-8663-5d85d7490ef3 service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:47.050610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 699/1371] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:40:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:40:47.656888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36532c03-0ba6-4c05-90f7-f6ad87449040 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:47.657325 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36532c03-0ba6-4c05-90f7-f6ad87449040 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:47.657443 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36532c03-0ba6-4c05-90f7-f6ad87449040 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:47.659318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 673/1372] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:40:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:47.665158 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7cf2b-5082-4e91-ba20-4cb79a85074e None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:47.665449 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7cf2b-5082-4e91-ba20-4cb79a85074e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:47.665757 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7cf2b-5082-4e91-ba20-4cb79a85074e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:47.684173 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-10f7cf2b-5082-4e91-ba20-4cb79a85074e None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:47.685392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-10f7cf2b-5082-4e91-ba20-4cb79a85074e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:47.693607 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 700/1373] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:40:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:48.046749 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da74dfc-f1de-4494-9b52-8a74f12aa4cb None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:48.046749 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da74dfc-f1de-4494-9b52-8a74f12aa4cb None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:48.046749 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da74dfc-f1de-4494-9b52-8a74f12aa4cb None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:48.048940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 674/1374] 10.222.0.20 () {54 vars in 963 bytes} [Mon Jun 27 15:40:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:48.054448 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74672579-7fd4-456f-a2b1-f96b949cb86d None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:48.054923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74672579-7fd4-456f-a2b1-f96b949cb86d None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:48.055005 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74672579-7fd4-456f-a2b1-f96b949cb86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:48.114347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-74672579-7fd4-456f-a2b1-f96b949cb86d None None] MFA Rules not processed for user `51404afd35b94c47ba5c44f0deb73e30`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:48.115586 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-74672579-7fd4-456f-a2b1-f96b949cb86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:48.120846 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:40:48.191794 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 701/1375] 10.222.0.20 () {58 vars in 1060 bytes} [Mon Jun 27 15:40:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:48.201607 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da74dfc-f1de-4494-9b52-8a74f12aa4cb None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:40:48.214195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da74dfc-f1de-4494-9b52-8a74f12aa4cb None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:48.218361 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a41f8-8ab1-436a-a1b7-53e678d83617 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:48.219062 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a41f8-8ab1-436a-a1b7-53e678d83617 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:48.219348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a41f8-8ab1-436a-a1b7-53e678d83617 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:48.219630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a41f8-8ab1-436a-a1b7-53e678d83617 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:48.230642 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a41f8-8ab1-436a-a1b7-53e678d83617 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:48.232385 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a41f8-8ab1-436a-a1b7-53e678d83617 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:48.243938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 675/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:40:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:40:49.182600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65929084-f661-4a30-9a60-0e217678c7b3 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:49.183177 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65929084-f661-4a30-9a60-0e217678c7b3 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:49.183177 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65929084-f661-4a30-9a60-0e217678c7b3 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:49.184565 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 702/1377] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:49.190223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ed540-bf82-4d29-baaa-3db1b047c86e None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:49.190767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ed540-bf82-4d29-baaa-3db1b047c86e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:49.190867 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ed540-bf82-4d29-baaa-3db1b047c86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:49.208980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-5f3ed540-bf82-4d29-baaa-3db1b047c86e None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:49.210124 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-5f3ed540-bf82-4d29-baaa-3db1b047c86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:49.218815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 676/1378] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:49.263183 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65929084-f661-4a30-9a60-0e217678c7b3 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:40:49.281662 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65929084-f661-4a30-9a60-0e217678c7b3 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:49.287124 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6b602-9aeb-4069-bdd3-5cbc6c920b65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:49.287776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6b602-9aeb-4069-bdd3-5cbc6c920b65 service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:49.288065 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6b602-9aeb-4069-bdd3-5cbc6c920b65 service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:49.288350 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6b602-9aeb-4069-bdd3-5cbc6c920b65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:49.298117 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6b602-9aeb-4069-bdd3-5cbc6c920b65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:49.299719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6b602-9aeb-4069-bdd3-5cbc6c920b65 service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:49.309602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 703/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:40:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:40:51.438333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477396e5-5297-4d09-9e4e-f7cbe45267ed None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:51.438837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477396e5-5297-4d09-9e4e-f7cbe45267ed None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:51.438837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477396e5-5297-4d09-9e4e-f7cbe45267ed None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:51.440200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 677/1380] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:40:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:51.445919 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce399269-1552-416e-8023-74bf0cee14cc None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:51.446390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce399269-1552-416e-8023-74bf0cee14cc None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:51.446483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce399269-1552-416e-8023-74bf0cee14cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:51.465616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-ce399269-1552-416e-8023-74bf0cee14cc None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:51.466853 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-ce399269-1552-416e-8023-74bf0cee14cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:51.475451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 704/1381] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:51.521920 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477396e5-5297-4d09-9e4e-f7cbe45267ed None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:40:51.536785 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477396e5-5297-4d09-9e4e-f7cbe45267ed None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:51.542453 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602dfba-2686-417d-ba04-d1eeff2ab4a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:51.543094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602dfba-2686-417d-ba04-d1eeff2ab4a8 service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:51.543379 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602dfba-2686-417d-ba04-d1eeff2ab4a8 service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:51.543658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602dfba-2686-417d-ba04-d1eeff2ab4a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:51.553101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602dfba-2686-417d-ba04-d1eeff2ab4a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:51.554736 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602dfba-2686-417d-ba04-d1eeff2ab4a8 service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:51.565725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 678/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:40:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:40:53.695911 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf6536-6b8f-4936-8d06-cb0581a2fb25 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:53.696450 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf6536-6b8f-4936-8d06-cb0581a2fb25 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:53.696450 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf6536-6b8f-4936-8d06-cb0581a2fb25 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:53.697869 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 705/1383] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:53.703923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81940e40-4a30-4ed2-8894-d790df926b1b None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:53.704386 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81940e40-4a30-4ed2-8894-d790df926b1b None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:53.704484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81940e40-4a30-4ed2-8894-d790df926b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:53.725310 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-81940e40-4a30-4ed2-8894-d790df926b1b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:53.726549 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-81940e40-4a30-4ed2-8894-d790df926b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:53.736840 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 679/1384] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:53.781465 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf6536-6b8f-4936-8d06-cb0581a2fb25 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:40:53.797450 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf6536-6b8f-4936-8d06-cb0581a2fb25 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:53.803928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4aeaf-b7a8-4547-b16f-2777c3d36fa3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:53.804612 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4aeaf-b7a8-4547-b16f-2777c3d36fa3 service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:53.804895 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4aeaf-b7a8-4547-b16f-2777c3d36fa3 service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:53.805169 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4aeaf-b7a8-4547-b16f-2777c3d36fa3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:53.814906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e4aeaf-b7a8-4547-b16f-2777c3d36fa3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:53.816825 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e4aeaf-b7a8-4547-b16f-2777c3d36fa3 service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:53.828291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 706/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:40:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:40:55.974597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30bcdcb-c7b1-4073-a074-78494377c9dd None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:55.975109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30bcdcb-c7b1-4073-a074-78494377c9dd None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:55.975109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30bcdcb-c7b1-4073-a074-78494377c9dd None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:55.976427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 680/1386] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:55.981987 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc3a8f-b2c8-43ba-b2c3-7d4f751abff9 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:55.982434 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc3a8f-b2c8-43ba-b2c3-7d4f751abff9 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:55.982690 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc3a8f-b2c8-43ba-b2c3-7d4f751abff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:56.002558 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-b6fc3a8f-b2c8-43ba-b2c3-7d4f751abff9 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:56.003784 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-b6fc3a8f-b2c8-43ba-b2c3-7d4f751abff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:56.011984 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 707/1387] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:56.060914 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30bcdcb-c7b1-4073-a074-78494377c9dd None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:40:56.075659 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30bcdcb-c7b1-4073-a074-78494377c9dd None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:56.082388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd38d8-6c97-48c3-95c4-f611993cf37c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:56.083043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd38d8-6c97-48c3-95c4-f611993cf37c service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:56.083320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd38d8-6c97-48c3-95c4-f611993cf37c service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:56.083605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd38d8-6c97-48c3-95c4-f611993cf37c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:56.092438 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd38d8-6c97-48c3-95c4-f611993cf37c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:56.094631 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd38d8-6c97-48c3-95c4-f611993cf37c service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:56.104615 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 681/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:40:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:40:58.222285 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43b6c2-bccc-4cb8-9da0-53e0bd42296c None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:58.222836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43b6c2-bccc-4cb8-9da0-53e0bd42296c None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:58.222836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43b6c2-bccc-4cb8-9da0-53e0bd42296c None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:58.224234 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 708/1389] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:40:58.230171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca95c0-653b-44bb-8961-ad90624e50db None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:58.230688 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca95c0-653b-44bb-8961-ad90624e50db None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:58.230905 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca95c0-653b-44bb-8961-ad90624e50db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:58.255967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-76ca95c0-653b-44bb-8961-ad90624e50db None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:40:58.257666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-76ca95c0-653b-44bb-8961-ad90624e50db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:40:58.268492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 682/1390] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:40:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:40:58.317822 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43b6c2-bccc-4cb8-9da0-53e0bd42296c None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:40:58.333211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43b6c2-bccc-4cb8-9da0-53e0bd42296c None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:40:58.338646 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d635f9e-2081-41b4-ac04-23ecb65aecfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:40:58.339293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d635f9e-2081-41b4-ac04-23ecb65aecfb service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:40:58.339573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d635f9e-2081-41b4-ac04-23ecb65aecfb service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:40:58.339847 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d635f9e-2081-41b4-ac04-23ecb65aecfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:40:58.348666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d635f9e-2081-41b4-ac04-23ecb65aecfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:40:58.350298 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d635f9e-2081-41b4-ac04-23ecb65aecfb service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:40:58.360577 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 709/1391] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:40:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:00.739128 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222c02a-4c98-42a3-a658-3a9434c92758 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:00.739731 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222c02a-4c98-42a3-a658-3a9434c92758 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:00.739731 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222c02a-4c98-42a3-a658-3a9434c92758 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:00.741056 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 683/1392] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:41:00.746610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175a486-797d-4b10-ac52-9763fd8eeb9a None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:00.747121 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175a486-797d-4b10-ac52-9763fd8eeb9a None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:00.747228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175a486-797d-4b10-ac52-9763fd8eeb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:00.768383 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-a175a486-797d-4b10-ac52-9763fd8eeb9a None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:00.769596 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-a175a486-797d-4b10-ac52-9763fd8eeb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:00.777842 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 710/1393] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:41:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:00.823378 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222c02a-4c98-42a3-a658-3a9434c92758 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:00.840001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222c02a-4c98-42a3-a658-3a9434c92758 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:00.846657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9cb75-c46b-4dc1-9314-84d55210fe7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:00.847322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9cb75-c46b-4dc1-9314-84d55210fe7b service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:00.847607 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9cb75-c46b-4dc1-9314-84d55210fe7b service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:00.847922 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9cb75-c46b-4dc1-9314-84d55210fe7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:00.856899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9cb75-c46b-4dc1-9314-84d55210fe7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:00.859457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9cb75-c46b-4dc1-9314-84d55210fe7b service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:00.869826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 684/1394] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:03.009117 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847f584-383d-45d7-92b4-7de305c09ac8 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:03.009751 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847f584-383d-45d7-92b4-7de305c09ac8 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:03.009751 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847f584-383d-45d7-92b4-7de305c09ac8 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:03.011103 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 711/1395] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:03.017147 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c1e4b-3228-4836-93a7-b5f62156ca7d None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:03.017711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c1e4b-3228-4836-93a7-b5f62156ca7d None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:03.017828 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c1e4b-3228-4836-93a7-b5f62156ca7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:03.036753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-d72c1e4b-3228-4836-93a7-b5f62156ca7d None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:03.037991 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d72c1e4b-3228-4836-93a7-b5f62156ca7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:03.047451 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 685/1396] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:03.094285 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847f584-383d-45d7-92b4-7de305c09ac8 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:03.109662 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847f584-383d-45d7-92b4-7de305c09ac8 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:03.115132 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32191c-73c1-4ecd-a9b3-18e5a1db9964 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:03.115788 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32191c-73c1-4ecd-a9b3-18e5a1db9964 service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:03.116074 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32191c-73c1-4ecd-a9b3-18e5a1db9964 service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:03.116351 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32191c-73c1-4ecd-a9b3-18e5a1db9964 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:03.126186 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32191c-73c1-4ecd-a9b3-18e5a1db9964 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:03.127811 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32191c-73c1-4ecd-a9b3-18e5a1db9964 service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:03.138159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 712/1397] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:03.611068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12131cb9-7d62-4179-bf39-1d8e5dc47d08 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:03.611358 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12131cb9-7d62-4179-bf39-1d8e5dc47d08 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:03.611637 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12131cb9-7d62-4179-bf39-1d8e5dc47d08 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:03.613092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 686/1398] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:03.618796 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d014c-ca28-426a-bca2-0b4b5e68cd9e None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:03.619081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d014c-ca28-426a-bca2-0b4b5e68cd9e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:03.619356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d014c-ca28-426a-bca2-0b4b5e68cd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:03.637506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-1e6d014c-ca28-426a-bca2-0b4b5e68cd9e None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:03.638745 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1e6d014c-ca28-426a-bca2-0b4b5e68cd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:03.647531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 713/1399] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:03.706420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcba17-249e-48ff-9b2a-a95d4841ab25 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:03.706718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcba17-249e-48ff-9b2a-a95d4841ab25 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:03.706997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dcba17-249e-48ff-9b2a-a95d4841ab25 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:03.708409 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 687/1400] 10.222.0.20 () {54 vars in 962 bytes} [Mon Jun 27 15:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:03.713763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e7d4f-c12a-4f0f-b3dd-4182d34ee643 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:03.714057 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e7d4f-c12a-4f0f-b3dd-4182d34ee643 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:03.714338 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e7d4f-c12a-4f0f-b3dd-4182d34ee643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:03.767654 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-eb2e7d4f-c12a-4f0f-b3dd-4182d34ee643 None None] MFA Rules not processed for user `493fcde4a45944efbc0b52c44e61b411`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:03.768901 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-eb2e7d4f-c12a-4f0f-b3dd-4182d34ee643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:03.840056 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 714/1401] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Jun 27 15:41:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:03.848450 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcba17-249e-48ff-9b2a-a95d4841ab25 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:03.853991 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=55 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:41:03.861579 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dcba17-249e-48ff-9b2a-a95d4841ab25 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:03.865002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cfe7d5-416e-42fe-8832-c06a569a1d78 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493fcde4a45944efbc0b52c44e61b411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:03.865669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfe7d5-416e-42fe-8832-c06a569a1d78 service glance] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:03.865966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfe7d5-416e-42fe-8832-c06a569a1d78 service glance] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:03.866242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfe7d5-416e-42fe-8832-c06a569a1d78 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:03.875971 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfe7d5-416e-42fe-8832-c06a569a1d78 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:03.877402 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cfe7d5-416e-42fe-8832-c06a569a1d78 service glance] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:03.887649 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 688/1402] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:04.654562 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654a565-4e73-4348-924b-40df5ab29377 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:04.655259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654a565-4e73-4348-924b-40df5ab29377 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:04.655259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654a565-4e73-4348-924b-40df5ab29377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:04.672080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:41:04.719573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-3654a565-4e73-4348-924b-40df5ab29377 None None] MFA Rules not processed for user `6655366e7eb84a26aa3f7fbe88af62b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:04.720904 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-3654a565-4e73-4348-924b-40df5ab29377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:04.815096 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 715/1403] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jun 27 15:41:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:41:04.869915 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1135b-cff1-4656-b5d5-7f42f573a53d None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:04.870492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1135b-cff1-4656-b5d5-7f42f573a53d None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:04.870614 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1135b-cff1-4656-b5d5-7f42f573a53d None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:04.872494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 689/1404] 10.222.0.20 () {54 vars in 942 bytes} [Mon Jun 27 15:41:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:04.880317 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21a146-64c9-4327-ace7-e064cd3d81a7 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:04.880674 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21a146-64c9-4327-ace7-e064cd3d81a7 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:04.881022 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21a146-64c9-4327-ace7-e064cd3d81a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:04.945238 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-7b21a146-64c9-4327-ace7-e064cd3d81a7 None None] MFA Rules not processed for user `8e83ea69ded040448fe75caa221ad377`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:04.946487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-7b21a146-64c9-4327-ace7-e064cd3d81a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:05.038449 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 716/1405] 10.222.0.20 () {58 vars in 1039 bytes} [Mon Jun 27 15:41:04 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:05.049502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1135b-cff1-4656-b5d5-7f42f573a53d None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:05.067050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1135b-cff1-4656-b5d5-7f42f573a53d None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:05.072404 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd5a1a-8137-4579-a36e-47d05797fb66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e83ea69ded040448fe75caa221ad377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:05.073098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd5a1a-8137-4579-a36e-47d05797fb66 service swift] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:05.073504 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd5a1a-8137-4579-a36e-47d05797fb66 service swift] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:05.073822 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd5a1a-8137-4579-a36e-47d05797fb66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:05.084439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd5a1a-8137-4579-a36e-47d05797fb66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:05.086280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd5a1a-8137-4579-a36e-47d05797fb66 service swift] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:05.096263 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 690/1406] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jun 27 15:41:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:41:05.279683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303d258-9d44-4a69-ba0c-6f8b91197952 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:05.280479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303d258-9d44-4a69-ba0c-6f8b91197952 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:05.281043 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303d258-9d44-4a69-ba0c-6f8b91197952 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:05.283964 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 717/1407] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:41:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:05.289265 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb16cc6-0f7e-4c26-a3b6-4892f3c68a26 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:05.289819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb16cc6-0f7e-4c26-a3b6-4892f3c68a26 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:05.290532 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb16cc6-0f7e-4c26-a3b6-4892f3c68a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:05.313655 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-7bb16cc6-0f7e-4c26-a3b6-4892f3c68a26 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:05.315130 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-7bb16cc6-0f7e-4c26-a3b6-4892f3c68a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:05.325692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 691/1408] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:41:05.375688 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303d258-9d44-4a69-ba0c-6f8b91197952 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:05.398346 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303d258-9d44-4a69-ba0c-6f8b91197952 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:05.413516 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27089d7a-9aca-4726-ba02-f34abf0b6837 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:05.414695 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27089d7a-9aca-4726-ba02-f34abf0b6837 service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:05.415201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27089d7a-9aca-4726-ba02-f34abf0b6837 service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:05.416045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27089d7a-9aca-4726-ba02-f34abf0b6837 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:05.428234 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27089d7a-9aca-4726-ba02-f34abf0b6837 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:05.429975 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27089d7a-9aca-4726-ba02-f34abf0b6837 service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:05.445487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 718/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:15.304709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=14 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:41:15.438765 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=87 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:41:28.171241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a87a9-5e4d-49af-a852-38765bebc82e None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:28.171906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a87a9-5e4d-49af-a852-38765bebc82e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:28.171906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a87a9-5e4d-49af-a852-38765bebc82e None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:28.173176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 692/1410] 10.222.0.20 () {54 vars in 918 bytes} [Mon Jun 27 15:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:28.178372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a376641-c879-4a39-9dce-e1dd2bdc385c None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:28.178846 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a376641-c879-4a39-9dce-e1dd2bdc385c None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:28.178955 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a376641-c879-4a39-9dce-e1dd2bdc385c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:28.197064 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-2a376641-c879-4a39-9dce-e1dd2bdc385c None None] MFA Rules not processed for user `ffafb68f1bb941588a5217ad83ad907c`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:28.198319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2a376641-c879-4a39-9dce-e1dd2bdc385c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:28.207511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 719/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Mon Jun 27 15:41:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:41:28.222885 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2bbbf-8b58-4d73-98c8-8353ca8edf4e None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:28.223177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2bbbf-8b58-4d73-98c8-8353ca8edf4e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:28.223459 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2bbbf-8b58-4d73-98c8-8353ca8edf4e None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:28.224828 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 693/1412] 10.222.0.20 () {54 vars in 918 bytes} [Mon Jun 27 15:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:28.229700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0002a-099b-4773-9c87-6e3db5238fd0 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:28.229989 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0002a-099b-4773-9c87-6e3db5238fd0 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:28.230296 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0002a-099b-4773-9c87-6e3db5238fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:28.252743 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-fbd0002a-099b-4773-9c87-6e3db5238fd0 None None] MFA Rules not processed for user `1c8f6262cf24419daf803ae71aed24f5`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:28.254002 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-fbd0002a-099b-4773-9c87-6e3db5238fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:28.263311 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 720/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Mon Jun 27 15:41:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:30.833769 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bf294-c22f-4e2d-8e91-8d996408904f None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:30.834269 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bf294-c22f-4e2d-8e91-8d996408904f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:30.834388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bf294-c22f-4e2d-8e91-8d996408904f None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:30.835731 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 694/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:30.841056 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05efc75c-8b3b-4200-ae10-f70be73c6190 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:30.841482 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05efc75c-8b3b-4200-ae10-f70be73c6190 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:30.841614 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05efc75c-8b3b-4200-ae10-f70be73c6190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:30.863590 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-05efc75c-8b3b-4200-ae10-f70be73c6190 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:30.864795 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-05efc75c-8b3b-4200-ae10-f70be73c6190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:30.872911 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 721/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:30.917193 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bf294-c22f-4e2d-8e91-8d996408904f None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:30.932043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bf294-c22f-4e2d-8e91-8d996408904f None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:30.938452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a34b2-2fbe-423f-a74f-62f107dd9631 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:30.939098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a34b2-2fbe-423f-a74f-62f107dd9631 service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:30.939384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a34b2-2fbe-423f-a74f-62f107dd9631 service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:30.939661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a34b2-2fbe-423f-a74f-62f107dd9631 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:30.947842 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a34b2-2fbe-423f-a74f-62f107dd9631 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:30.949466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a34b2-2fbe-423f-a74f-62f107dd9631 service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:30.959616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 695/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:31.897314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2141d-77bd-4540-8241-966cbf83f66e None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:31.897874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2141d-77bd-4540-8241-966cbf83f66e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:31.897874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2141d-77bd-4540-8241-966cbf83f66e None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:31.899295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 722/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:31.904367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205febd6-c395-476c-a2de-d0bb0dd82fb8 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:31.904807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205febd6-c395-476c-a2de-d0bb0dd82fb8 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:31.904917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205febd6-c395-476c-a2de-d0bb0dd82fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:31.923659 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-205febd6-c395-476c-a2de-d0bb0dd82fb8 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:31.924872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-205febd6-c395-476c-a2de-d0bb0dd82fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:31.932720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 696/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:32.085830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2141d-77bd-4540-8241-966cbf83f66e None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:32.100941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2141d-77bd-4540-8241-966cbf83f66e None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:32.106406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54fcfe-0ab6-4342-837b-5e8c9f11a7e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:32.107069 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54fcfe-0ab6-4342-837b-5e8c9f11a7e0 service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:32.107356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54fcfe-0ab6-4342-837b-5e8c9f11a7e0 service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:32.107636 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54fcfe-0ab6-4342-837b-5e8c9f11a7e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:32.117138 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54fcfe-0ab6-4342-837b-5e8c9f11a7e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:32.118840 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54fcfe-0ab6-4342-837b-5e8c9f11a7e0 service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:32.130683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 723/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:33.028439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75e724-3623-4e3d-94da-a061b8fa72f2 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:33.029016 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75e724-3623-4e3d-94da-a061b8fa72f2 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:33.029016 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75e724-3623-4e3d-94da-a061b8fa72f2 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:33.031049 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 697/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:33.036745 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e927b44-0cf5-4ae0-952d-17cb59eb6e67 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:33.037276 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e927b44-0cf5-4ae0-952d-17cb59eb6e67 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:33.037402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e927b44-0cf5-4ae0-952d-17cb59eb6e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:33.056863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-1e927b44-0cf5-4ae0-952d-17cb59eb6e67 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:33.058266 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1e927b44-0cf5-4ae0-952d-17cb59eb6e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:33.071426 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 724/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:33.225314 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75e724-3623-4e3d-94da-a061b8fa72f2 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:33.239996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75e724-3623-4e3d-94da-a061b8fa72f2 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:33.245238 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8800120-d75e-46f8-a328-1b113d258285 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:33.245911 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8800120-d75e-46f8-a328-1b113d258285 service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:33.246194 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8800120-d75e-46f8-a328-1b113d258285 service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:33.246471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8800120-d75e-46f8-a328-1b113d258285 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:33.254593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8800120-d75e-46f8-a328-1b113d258285 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:33.256161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8800120-d75e-46f8-a328-1b113d258285 service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:33.265275 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 698/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:34.144898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d5a3a-402c-4226-8b71-22d2b9281b2e None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:34.145473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d5a3a-402c-4226-8b71-22d2b9281b2e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:34.145473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d5a3a-402c-4226-8b71-22d2b9281b2e None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:34.146871 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 725/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:34.152172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30d891-fd16-4ce9-ab92-aeefc06f94e7 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:34.152627 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30d891-fd16-4ce9-ab92-aeefc06f94e7 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:34.152722 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30d891-fd16-4ce9-ab92-aeefc06f94e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:34.172115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-8b30d891-fd16-4ce9-ab92-aeefc06f94e7 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:34.173330 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-8b30d891-fd16-4ce9-ab92-aeefc06f94e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:34.181105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 699/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:34.333687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d5a3a-402c-4226-8b71-22d2b9281b2e None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:34.348534 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d5a3a-402c-4226-8b71-22d2b9281b2e None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:34.353931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03839be6-76a1-4f99-8288-5d5bf562cee5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:34.354574 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03839be6-76a1-4f99-8288-5d5bf562cee5 service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:34.354862 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03839be6-76a1-4f99-8288-5d5bf562cee5 service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:34.355141 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03839be6-76a1-4f99-8288-5d5bf562cee5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:34.363664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03839be6-76a1-4f99-8288-5d5bf562cee5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:34.365229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03839be6-76a1-4f99-8288-5d5bf562cee5 service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:34.374835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 726/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:35.257708 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dc023-cc97-4b79-ba4d-3b5f545690b8 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:35.258280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dc023-cc97-4b79-ba4d-3b5f545690b8 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:35.258280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dc023-cc97-4b79-ba4d-3b5f545690b8 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:35.259650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 700/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:35.264708 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1168251-3289-43c0-ad63-722bfa023225 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:35.265110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1168251-3289-43c0-ad63-722bfa023225 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:35.265326 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1168251-3289-43c0-ad63-722bfa023225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:35.283857 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-d1168251-3289-43c0-ad63-722bfa023225 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:35.285051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-d1168251-3289-43c0-ad63-722bfa023225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:35.293309 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 727/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:35.448147 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dc023-cc97-4b79-ba4d-3b5f545690b8 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:35.463574 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dc023-cc97-4b79-ba4d-3b5f545690b8 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:35.469414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db10ff7-15af-43aa-9a52-421096b13948 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:35.470093 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db10ff7-15af-43aa-9a52-421096b13948 service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:35.470397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db10ff7-15af-43aa-9a52-421096b13948 service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:35.470672 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db10ff7-15af-43aa-9a52-421096b13948 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:35.479447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db10ff7-15af-43aa-9a52-421096b13948 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:35.481022 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db10ff7-15af-43aa-9a52-421096b13948 service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:35.491330 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 701/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:36.335870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399f641-fecc-4455-9676-405757b44684 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:36.336447 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399f641-fecc-4455-9676-405757b44684 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:36.336447 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399f641-fecc-4455-9676-405757b44684 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:36.337830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 728/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:36.343517 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac737e1d-331d-4170-ac65-f045bac42954 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:36.343980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac737e1d-331d-4170-ac65-f045bac42954 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:36.344208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac737e1d-331d-4170-ac65-f045bac42954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:36.363266 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-ac737e1d-331d-4170-ac65-f045bac42954 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:36.364476 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-ac737e1d-331d-4170-ac65-f045bac42954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:36.372337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 702/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:41:36.524133 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399f641-fecc-4455-9676-405757b44684 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:36.538396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399f641-fecc-4455-9676-405757b44684 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:36.544010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd6d5d-2707-497f-aa76-431b1920d75c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:36.544652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd6d5d-2707-497f-aa76-431b1920d75c service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:36.544933 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd6d5d-2707-497f-aa76-431b1920d75c service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:36.545210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd6d5d-2707-497f-aa76-431b1920d75c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:36.554317 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd6d5d-2707-497f-aa76-431b1920d75c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:36.556050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd6d5d-2707-497f-aa76-431b1920d75c service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:36.566785 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 729/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:37.394536 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150dd734-5b5f-438e-96f2-cc8992a53c39 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:37.395234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150dd734-5b5f-438e-96f2-cc8992a53c39 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:37.395234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150dd734-5b5f-438e-96f2-cc8992a53c39 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:37.396482 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 703/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:37.401758 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c236c-ebb2-4251-9385-8c9e84d073c4 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:37.402247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c236c-ebb2-4251-9385-8c9e84d073c4 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:37.402359 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c236c-ebb2-4251-9385-8c9e84d073c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:37.421516 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-875c236c-ebb2-4251-9385-8c9e84d073c4 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:37.422733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-875c236c-ebb2-4251-9385-8c9e84d073c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:37.430495 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 730/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:41:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:37.589567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150dd734-5b5f-438e-96f2-cc8992a53c39 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:37.604116 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150dd734-5b5f-438e-96f2-cc8992a53c39 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:37.610242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778252f6-4185-4f5d-be0a-d9cccd9bdb78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:37.610890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778252f6-4185-4f5d-be0a-d9cccd9bdb78 service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:37.611175 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778252f6-4185-4f5d-be0a-d9cccd9bdb78 service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:37.611461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778252f6-4185-4f5d-be0a-d9cccd9bdb78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:37.620023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778252f6-4185-4f5d-be0a-d9cccd9bdb78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:37.621615 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778252f6-4185-4f5d-be0a-d9cccd9bdb78 service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:37.631968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 704/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:38.475452 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68655f5-f28a-4fa6-b76a-25a9ff427491 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:38.475991 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68655f5-f28a-4fa6-b76a-25a9ff427491 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:38.475991 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68655f5-f28a-4fa6-b76a-25a9ff427491 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:38.477844 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 731/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:38.483311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1570d-07b5-4f5b-af79-48cc4729b00f None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:38.483311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1570d-07b5-4f5b-af79-48cc4729b00f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:38.483311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1570d-07b5-4f5b-af79-48cc4729b00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:38.502711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-42a1570d-07b5-4f5b-af79-48cc4729b00f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:38.503938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-42a1570d-07b5-4f5b-af79-48cc4729b00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:38.511861 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 705/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:41:38.665705 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68655f5-f28a-4fa6-b76a-25a9ff427491 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:38.680492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68655f5-f28a-4fa6-b76a-25a9ff427491 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:38.685927 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0258df5-5522-4066-8351-186263ebdd6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:38.687214 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0258df5-5522-4066-8351-186263ebdd6d service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:38.687519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0258df5-5522-4066-8351-186263ebdd6d service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:38.687800 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0258df5-5522-4066-8351-186263ebdd6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:38.699179 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0258df5-5522-4066-8351-186263ebdd6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:38.700770 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0258df5-5522-4066-8351-186263ebdd6d service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:38.711559 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 732/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:39.552761 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a643a-def5-4627-8b41-43752c5be3df None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:39.553345 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a643a-def5-4627-8b41-43752c5be3df None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:39.553345 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a643a-def5-4627-8b41-43752c5be3df None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:39.554736 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 706/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:39.559788 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b06e1-5af4-4938-8f69-33e8491690ce None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:39.560255 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b06e1-5af4-4938-8f69-33e8491690ce None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:39.560381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b06e1-5af4-4938-8f69-33e8491690ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:39.579367 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-b43b06e1-5af4-4938-8f69-33e8491690ce None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:39.580709 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-b43b06e1-5af4-4938-8f69-33e8491690ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:39.588982 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 733/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:39.748662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a643a-def5-4627-8b41-43752c5be3df None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:39.763935 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a643a-def5-4627-8b41-43752c5be3df None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:39.770270 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c9f9f-ac83-4e47-b5d9-b6a0dc6ac850 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:39.770940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c9f9f-ac83-4e47-b5d9-b6a0dc6ac850 service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:39.771219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c9f9f-ac83-4e47-b5d9-b6a0dc6ac850 service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:39.771494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c9f9f-ac83-4e47-b5d9-b6a0dc6ac850 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:39.780239 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c9f9f-ac83-4e47-b5d9-b6a0dc6ac850 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:39.781861 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c9f9f-ac83-4e47-b5d9-b6a0dc6ac850 service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:39.791818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 707/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:41.739381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db8500-b195-4c51-bc9a-ef05c2d3c5a6 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:41.739945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db8500-b195-4c51-bc9a-ef05c2d3c5a6 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:41.739945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db8500-b195-4c51-bc9a-ef05c2d3c5a6 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:41.741319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 734/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:41.746815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb229a-4a3b-428a-b591-8980d89431e1 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:41.747250 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb229a-4a3b-428a-b591-8980d89431e1 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:41.747380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb229a-4a3b-428a-b591-8980d89431e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:41.766263 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-5dbb229a-4a3b-428a-b591-8980d89431e1 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:41.767478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-5dbb229a-4a3b-428a-b591-8980d89431e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:41.775320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 708/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:41.814880 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db8500-b195-4c51-bc9a-ef05c2d3c5a6 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:41.825916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db8500-b195-4c51-bc9a-ef05c2d3c5a6 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:41.830026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743bbaf-9b5d-4590-a992-8ee18e75d02a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:41.830676 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743bbaf-9b5d-4590-a992-8ee18e75d02a service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:41.830965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743bbaf-9b5d-4590-a992-8ee18e75d02a service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:41.831276 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743bbaf-9b5d-4590-a992-8ee18e75d02a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:41.840598 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3743bbaf-9b5d-4590-a992-8ee18e75d02a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:41.842057 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3743bbaf-9b5d-4590-a992-8ee18e75d02a service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:41.852376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 735/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:44.151508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67e117-116c-4f7c-ab45-3ad3bf21589a None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:44.152114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67e117-116c-4f7c-ab45-3ad3bf21589a None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:44.152114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67e117-116c-4f7c-ab45-3ad3bf21589a None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:44.153483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 709/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:44.159601 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e6414-9f4d-4dac-9969-3d7f24e3c15f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:44.160054 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e6414-9f4d-4dac-9969-3d7f24e3c15f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:44.160154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e6414-9f4d-4dac-9969-3d7f24e3c15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:44.178495 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-e08e6414-9f4d-4dac-9969-3d7f24e3c15f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:44.179691 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e08e6414-9f4d-4dac-9969-3d7f24e3c15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:44.187536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 736/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:41:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:44.227279 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67e117-116c-4f7c-ab45-3ad3bf21589a None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:44.229144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-dd67e117-116c-4f7c-ab45-3ad3bf21589a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:44.242891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67e117-116c-4f7c-ab45-3ad3bf21589a None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:44.246478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82212a-3524-41a3-ae22-5011fe75e91d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:44.247118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82212a-3524-41a3-ae22-5011fe75e91d service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:44.247399 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82212a-3524-41a3-ae22-5011fe75e91d service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:44.247676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82212a-3524-41a3-ae22-5011fe75e91d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:44.257966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82212a-3524-41a3-ae22-5011fe75e91d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:44.259482 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82212a-3524-41a3-ae22-5011fe75e91d service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:44.270895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 710/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:46.748584 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a984ad-794a-484f-b88f-1695cb773fbb None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:46.749705 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a984ad-794a-484f-b88f-1695cb773fbb None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:46.749705 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a984ad-794a-484f-b88f-1695cb773fbb None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:46.751041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 737/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:41:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:46.756463 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a72b3b-62cc-4096-9a38-f2b1728fa4d3 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:46.756883 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a72b3b-62cc-4096-9a38-f2b1728fa4d3 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:46.757006 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a72b3b-62cc-4096-9a38-f2b1728fa4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:46.775387 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-46a72b3b-62cc-4096-9a38-f2b1728fa4d3 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:46.776619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-46a72b3b-62cc-4096-9a38-f2b1728fa4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:46.784828 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 711/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:46.824223 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a984ad-794a-484f-b88f-1695cb773fbb None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:46.836138 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a984ad-794a-484f-b88f-1695cb773fbb None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:46.839796 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3cbf72-4441-4d44-be5c-899fbc738bd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:46.840440 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3cbf72-4441-4d44-be5c-899fbc738bd6 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:46.840727 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3cbf72-4441-4d44-be5c-899fbc738bd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:46.841009 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3cbf72-4441-4d44-be5c-899fbc738bd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:46.851252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3cbf72-4441-4d44-be5c-899fbc738bd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:46.852688 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3cbf72-4441-4d44-be5c-899fbc738bd6 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:46.862808 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 738/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:49.087846 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33659351-876b-4173-b2b9-f36330bb9e98 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:49.088379 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33659351-876b-4173-b2b9-f36330bb9e98 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:49.088500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33659351-876b-4173-b2b9-f36330bb9e98 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:49.089822 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 712/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:49.095716 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f89b64-adca-42dc-bbef-59f357bc9d84 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:49.096217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f89b64-adca-42dc-bbef-59f357bc9d84 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:49.096332 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f89b64-adca-42dc-bbef-59f357bc9d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:49.116232 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-56f89b64-adca-42dc-bbef-59f357bc9d84 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:49.117446 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-56f89b64-adca-42dc-bbef-59f357bc9d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:49.125942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 739/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:41:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:49.165281 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33659351-876b-4173-b2b9-f36330bb9e98 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:49.177185 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33659351-876b-4173-b2b9-f36330bb9e98 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:49.180689 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43189e-de31-4033-8a08-b1fa9591d094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:49.181354 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43189e-de31-4033-8a08-b1fa9591d094 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:49.181667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43189e-de31-4033-8a08-b1fa9591d094 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:49.181944 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43189e-de31-4033-8a08-b1fa9591d094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:49.191894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43189e-de31-4033-8a08-b1fa9591d094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:49.193427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43189e-de31-4033-8a08-b1fa9591d094 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:49.204384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 713/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:51.373414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89406cba-3b03-4ec1-9ddf-720a797f1aad None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:51.374046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89406cba-3b03-4ec1-9ddf-720a797f1aad None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:51.374493 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89406cba-3b03-4ec1-9ddf-720a797f1aad None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:51.375890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 740/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:41:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:51.381354 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbed20-4cdb-4a9d-881a-f291ffbca055 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:51.381858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbed20-4cdb-4a9d-881a-f291ffbca055 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:51.382028 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbed20-4cdb-4a9d-881a-f291ffbca055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:51.401141 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-efdbed20-4cdb-4a9d-881a-f291ffbca055 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:51.402621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-efdbed20-4cdb-4a9d-881a-f291ffbca055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:51.411609 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 714/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:51.450854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89406cba-3b03-4ec1-9ddf-720a797f1aad None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:51.462234 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89406cba-3b03-4ec1-9ddf-720a797f1aad None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:51.465724 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590bee0d-d916-46a1-b9fc-0884686e24b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:51.466359 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bee0d-d916-46a1-b9fc-0884686e24b2 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:51.466643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bee0d-d916-46a1-b9fc-0884686e24b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:51.466922 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bee0d-d916-46a1-b9fc-0884686e24b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:51.477398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590bee0d-d916-46a1-b9fc-0884686e24b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:51.478864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590bee0d-d916-46a1-b9fc-0884686e24b2 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:51.488792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 741/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:53.711665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2ad61-a5ed-45af-bd39-e73a2ef542d3 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:53.712249 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2ad61-a5ed-45af-bd39-e73a2ef542d3 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:53.712646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af2ad61-a5ed-45af-bd39-e73a2ef542d3 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:53.715209 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 715/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:41:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:53.721482 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e62b4c-f821-4c68-acfd-62d4c5e0b304 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:53.722339 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e62b4c-f821-4c68-acfd-62d4c5e0b304 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:53.722339 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e62b4c-f821-4c68-acfd-62d4c5e0b304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:53.742393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-c4e62b4c-f821-4c68-acfd-62d4c5e0b304 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:53.743630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-c4e62b4c-f821-4c68-acfd-62d4c5e0b304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:53.752267 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 742/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:53.793988 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af2ad61-a5ed-45af-bd39-e73a2ef542d3 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:53.806965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af2ad61-a5ed-45af-bd39-e73a2ef542d3 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:53.810970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbadf32-bd7c-4853-89ab-56baf863d1dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:53.812360 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbadf32-bd7c-4853-89ab-56baf863d1dc service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:53.812649 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbadf32-bd7c-4853-89ab-56baf863d1dc service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:53.812928 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbadf32-bd7c-4853-89ab-56baf863d1dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:53.824343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbadf32-bd7c-4853-89ab-56baf863d1dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:53.826054 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbadf32-bd7c-4853-89ab-56baf863d1dc service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:53.846548 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 716/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:56.194728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165695c3-d91a-4e22-b79e-f8c169fe9a04 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:56.195628 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165695c3-d91a-4e22-b79e-f8c169fe9a04 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:56.195628 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165695c3-d91a-4e22-b79e-f8c169fe9a04 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:56.196918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 743/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:41:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:41:56.205220 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0a974-434a-4523-bc2e-af668f1f1974 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:56.205925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0a974-434a-4523-bc2e-af668f1f1974 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:56.206039 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0a974-434a-4523-bc2e-af668f1f1974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:56.213725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=70 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:41:56.228274 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-64f0a974-434a-4523-bc2e-af668f1f1974 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:56.229702 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-64f0a974-434a-4523-bc2e-af668f1f1974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:56.239332 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 717/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:56.293857 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165695c3-d91a-4e22-b79e-f8c169fe9a04 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:56.302525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=73 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:41:56.310198 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165695c3-d91a-4e22-b79e-f8c169fe9a04 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:56.314722 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9304fe0-824d-4ec6-a5bf-6e35324f3f8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:56.315392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9304fe0-824d-4ec6-a5bf-6e35324f3f8a service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:56.315672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9304fe0-824d-4ec6-a5bf-6e35324f3f8a service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:56.315949 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9304fe0-824d-4ec6-a5bf-6e35324f3f8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:56.326965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9304fe0-824d-4ec6-a5bf-6e35324f3f8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:56.328504 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9304fe0-824d-4ec6-a5bf-6e35324f3f8a service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:56.339492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 744/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:41:58.662950 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b872c-b63f-4b80-9e05-ee8f7094adb1 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:58.663501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b872c-b63f-4b80-9e05-ee8f7094adb1 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:58.663501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b872c-b63f-4b80-9e05-ee8f7094adb1 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:58.664838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 718/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:41:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:41:58.672827 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052944b0-3bf6-4406-952d-a8655a782e39 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:58.673380 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052944b0-3bf6-4406-952d-a8655a782e39 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:58.673380 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052944b0-3bf6-4406-952d-a8655a782e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:58.693327 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-052944b0-3bf6-4406-952d-a8655a782e39 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:41:58.694602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-052944b0-3bf6-4406-952d-a8655a782e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:41:58.703385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 745/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:41:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:41:58.745738 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b872c-b63f-4b80-9e05-ee8f7094adb1 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:41:58.758335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b872c-b63f-4b80-9e05-ee8f7094adb1 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:41:58.762579 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f4f1c-5cca-4614-8226-bd4f490af948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:41:58.763299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f4f1c-5cca-4614-8226-bd4f490af948 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:41:58.763623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f4f1c-5cca-4614-8226-bd4f490af948 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:41:58.763938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f4f1c-5cca-4614-8226-bd4f490af948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:41:58.775628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f4f1c-5cca-4614-8226-bd4f490af948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:41:58.777087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f4f1c-5cca-4614-8226-bd4f490af948 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:41:58.788869 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 719/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:41:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:42:01.321182 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10dce0-73d2-421a-8f7a-3615902f8ec4 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:01.321798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10dce0-73d2-421a-8f7a-3615902f8ec4 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:01.321798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10dce0-73d2-421a-8f7a-3615902f8ec4 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:01.323259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 746/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:42:01.330455 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e69338-537e-44e8-bd55-ab3e9edb855d None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:01.331006 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e69338-537e-44e8-bd55-ab3e9edb855d None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:01.331110 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e69338-537e-44e8-bd55-ab3e9edb855d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:01.354010 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-a8e69338-537e-44e8-bd55-ab3e9edb855d None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:42:01.355396 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a8e69338-537e-44e8-bd55-ab3e9edb855d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:42:01.365916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 720/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:42:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:42:01.410984 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10dce0-73d2-421a-8f7a-3615902f8ec4 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:42:01.423874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10dce0-73d2-421a-8f7a-3615902f8ec4 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:42:01.428314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7914fd-e9c0-42f5-8696-6e8d38c5080a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:42:01.428998 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7914fd-e9c0-42f5-8696-6e8d38c5080a service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:01.429282 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7914fd-e9c0-42f5-8696-6e8d38c5080a service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:01.429591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7914fd-e9c0-42f5-8696-6e8d38c5080a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:01.442178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7914fd-e9c0-42f5-8696-6e8d38c5080a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:42:01.443777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7914fd-e9c0-42f5-8696-6e8d38c5080a service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:42:01.455785 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 747/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:42:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:42:03.655452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9afb77-2fe7-4233-bea9-e1507b788a3e None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:03.656235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9afb77-2fe7-4233-bea9-e1507b788a3e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:03.656365 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9afb77-2fe7-4233-bea9-e1507b788a3e None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:03.658899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 721/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:42:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:42:03.666295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae832ed-af6a-4cd5-a28d-773a8bd960e6 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:03.667384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae832ed-af6a-4cd5-a28d-773a8bd960e6 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:03.667560 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae832ed-af6a-4cd5-a28d-773a8bd960e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:03.688965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-1ae832ed-af6a-4cd5-a28d-773a8bd960e6 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:42:03.690262 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1ae832ed-af6a-4cd5-a28d-773a8bd960e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:42:03.698734 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 748/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:42:03.737759 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9afb77-2fe7-4233-bea9-e1507b788a3e None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:42:03.751017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9afb77-2fe7-4233-bea9-e1507b788a3e None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:42:03.755232 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed8471-e49b-4973-a6cb-aa0da3521538 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:42:03.755933 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed8471-e49b-4973-a6cb-aa0da3521538 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:03.756248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed8471-e49b-4973-a6cb-aa0da3521538 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:03.756561 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed8471-e49b-4973-a6cb-aa0da3521538 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:03.767601 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed8471-e49b-4973-a6cb-aa0da3521538 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:42:03.769062 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed8471-e49b-4973-a6cb-aa0da3521538 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:42:03.779841 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 722/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:42:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:42:13.681422 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=17 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:42:13.775048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=17 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:42:25.757625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b188d6a-ec24-4ccd-90cf-1c18b350b4c0 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:25.758229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b188d6a-ec24-4ccd-90cf-1c18b350b4c0 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:25.758229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b188d6a-ec24-4ccd-90cf-1c18b350b4c0 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:25.759623 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 749/1471] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:42:25.765187 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add960e1-7581-41b5-85c5-10a5e0960d07 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:25.765682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add960e1-7581-41b5-85c5-10a5e0960d07 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:25.765819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add960e1-7581-41b5-85c5-10a5e0960d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:25.783672 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-add960e1-7581-41b5-85c5-10a5e0960d07 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:42:25.784865 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-add960e1-7581-41b5-85c5-10a5e0960d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:42:25.792803 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 723/1472] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:42:25.840243 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdbbf0-790a-4f9b-975c-27b9c5705068 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:25.840534 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdbbf0-790a-4f9b-975c-27b9c5705068 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:25.840813 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bdbbf0-790a-4f9b-975c-27b9c5705068 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:25.842767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 750/1473] 10.222.0.20 () {54 vars in 962 bytes} [Mon Jun 27 15:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:42:25.848464 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9015db-2d84-4584-b024-1c3ba5d65915 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:25.848752 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9015db-2d84-4584-b024-1c3ba5d65915 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:25.849026 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9015db-2d84-4584-b024-1c3ba5d65915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:25.867599 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-dc9015db-2d84-4584-b024-1c3ba5d65915 None None] MFA Rules not processed for user `493fcde4a45944efbc0b52c44e61b411`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:42:25.868784 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-dc9015db-2d84-4584-b024-1c3ba5d65915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:42:25.875235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 724/1474] 10.222.0.20 () {58 vars in 1059 bytes} [Mon Jun 27 15:42:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:42:25.882775 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdbbf0-790a-4f9b-975c-27b9c5705068 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:42:25.894417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bdbbf0-790a-4f9b-975c-27b9c5705068 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:42:25.898005 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f49db8-ad6d-479f-bed3-b498da47ac51 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493fcde4a45944efbc0b52c44e61b411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:42:25.898646 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f49db8-ad6d-479f-bed3-b498da47ac51 service glance] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:25.898925 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f49db8-ad6d-479f-bed3-b498da47ac51 service glance] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:25.899193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f49db8-ad6d-479f-bed3-b498da47ac51 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:25.908775 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f49db8-ad6d-479f-bed3-b498da47ac51 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:42:25.910235 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f49db8-ad6d-479f-bed3-b498da47ac51 service glance] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:42:25.919959 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 751/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:42:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:42:28.104860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66deb2a3-d4bf-45e4-81c0-c57fb6cd3a2d None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:28.105607 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66deb2a3-d4bf-45e4-81c0-c57fb6cd3a2d None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:28.105607 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66deb2a3-d4bf-45e4-81c0-c57fb6cd3a2d None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:28.106813 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 725/1476] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:42:28.112804 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bd60d-a67d-4dfb-9714-4c4a099df916 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:28.113270 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bd60d-a67d-4dfb-9714-4c4a099df916 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:28.113380 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bd60d-a67d-4dfb-9714-4c4a099df916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:28.135282 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-ce7bd60d-a67d-4dfb-9714-4c4a099df916 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:42:28.136489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-ce7bd60d-a67d-4dfb-9714-4c4a099df916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:42:28.144362 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 752/1477] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:42:28.168986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66deb2a3-d4bf-45e4-81c0-c57fb6cd3a2d None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:42:28.182993 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66deb2a3-d4bf-45e4-81c0-c57fb6cd3a2d None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:42:28.188232 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fd455-b62b-44ca-9ef4-21178a125c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:42:28.188891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fd455-b62b-44ca-9ef4-21178a125c2a admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:28.189173 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fd455-b62b-44ca-9ef4-21178a125c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:28.189449 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fd455-b62b-44ca-9ef4-21178a125c2a admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:28.190220 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fd455-b62b-44ca-9ef4-21178a125c2a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:42:28.192011 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: WARNING py.warnings [None req-872fd455-b62b-44ca-9ef4-21178a125c2a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:42:28.192011 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: warnings.warn(msg) Jun 27 15:42:28.192011 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: Jun 27 15:42:28.193157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fd455-b62b-44ca-9ef4-21178a125c2a admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:42:28.212784 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 726/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Mon Jun 27 15:42:28 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:42:30.322439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7835b-c9c4-4364-9168-f5a0d1ecc2bd None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:30.323013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7835b-c9c4-4364-9168-f5a0d1ecc2bd None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:30.323013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7835b-c9c4-4364-9168-f5a0d1ecc2bd None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:30.324373 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 753/1479] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:42:30.329680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fd5c6-da45-43f9-bfef-636f9f88afe0 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:30.330131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fd5c6-da45-43f9-bfef-636f9f88afe0 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:30.330343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fd5c6-da45-43f9-bfef-636f9f88afe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:30.348871 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-a19fd5c6-da45-43f9-bfef-636f9f88afe0 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:42:30.350114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a19fd5c6-da45-43f9-bfef-636f9f88afe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:42:30.358587 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 727/1480] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:42:30.402904 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7835b-c9c4-4364-9168-f5a0d1ecc2bd None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:42:30.419030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7835b-c9c4-4364-9168-f5a0d1ecc2bd None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:42:30.424634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df806232-ec8a-4c77-832a-da75825c9271 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:42:30.425274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df806232-ec8a-4c77-832a-da75825c9271 service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:30.425560 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df806232-ec8a-4c77-832a-da75825c9271 service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:30.425893 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df806232-ec8a-4c77-832a-da75825c9271 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:30.434203 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df806232-ec8a-4c77-832a-da75825c9271 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:42:30.435792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df806232-ec8a-4c77-832a-da75825c9271 service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:42:30.445727 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 754/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:42:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:42:32.716487 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdb1e2-f25a-40b6-9802-939bf0bd8729 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:32.717060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdb1e2-f25a-40b6-9802-939bf0bd8729 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:32.717060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdb1e2-f25a-40b6-9802-939bf0bd8729 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:32.718575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 728/1482] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:42:32.723744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b50c05-be3d-48e3-9f6f-f2e6282f7f90 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:32.724202 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b50c05-be3d-48e3-9f6f-f2e6282f7f90 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:32.724300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b50c05-be3d-48e3-9f6f-f2e6282f7f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:32.742586 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-26b50c05-be3d-48e3-9f6f-f2e6282f7f90 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:42:32.743805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-26b50c05-be3d-48e3-9f6f-f2e6282f7f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:42:32.752230 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 755/1483] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:42:32.789353 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bdb1e2-f25a-40b6-9802-939bf0bd8729 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:42:32.800980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bdb1e2-f25a-40b6-9802-939bf0bd8729 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:42:32.804547 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f98ce6-412d-4d79-bf35-e239e7b4e788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:42:32.805198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f98ce6-412d-4d79-bf35-e239e7b4e788 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:32.805486 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f98ce6-412d-4d79-bf35-e239e7b4e788 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:32.805807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f98ce6-412d-4d79-bf35-e239e7b4e788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:32.815747 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f98ce6-412d-4d79-bf35-e239e7b4e788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:42:32.817289 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f98ce6-412d-4d79-bf35-e239e7b4e788 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:42:32.828114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 729/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:42:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:42:34.994343 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ae0e7-20c4-4a07-8299-0bb734297f66 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:34.994915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ae0e7-20c4-4a07-8299-0bb734297f66 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:34.994915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ae0e7-20c4-4a07-8299-0bb734297f66 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:34.996261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 756/1485] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:42:35.002276 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b74f4-9b5b-4316-909f-b2a48bb2eaae None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:35.002705 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b74f4-9b5b-4316-909f-b2a48bb2eaae None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:35.002829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b74f4-9b5b-4316-909f-b2a48bb2eaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:35.020606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-f35b74f4-9b5b-4316-909f-b2a48bb2eaae None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:42:35.021835 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f35b74f4-9b5b-4316-909f-b2a48bb2eaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:42:35.029962 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 730/1486] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:42:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:42:35.068151 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ae0e7-20c4-4a07-8299-0bb734297f66 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:42:35.079135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ae0e7-20c4-4a07-8299-0bb734297f66 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:42:35.082577 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eebab0-596e-4166-a164-6d84cd3b2de5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:42:35.083207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eebab0-596e-4166-a164-6d84cd3b2de5 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:35.083484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eebab0-596e-4166-a164-6d84cd3b2de5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:35.083755 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eebab0-596e-4166-a164-6d84cd3b2de5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:35.092886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eebab0-596e-4166-a164-6d84cd3b2de5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:42:35.094347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eebab0-596e-4166-a164-6d84cd3b2de5 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:42:35.103518 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 757/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:42:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:42:37.366205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051c11e-8db0-4cb4-8d23-6f0a06eb38bf None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:37.366743 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051c11e-8db0-4cb4-8d23-6f0a06eb38bf None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:37.366743 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051c11e-8db0-4cb4-8d23-6f0a06eb38bf None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:37.368118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 731/1488] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:42:37.374213 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5055a7-a018-426d-8c1b-a3c619f43b9b None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:37.374938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5055a7-a018-426d-8c1b-a3c619f43b9b None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:37.375222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5055a7-a018-426d-8c1b-a3c619f43b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:37.393016 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-7c5055a7-a018-426d-8c1b-a3c619f43b9b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:42:37.394253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-7c5055a7-a018-426d-8c1b-a3c619f43b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:42:37.402062 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 758/1489] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:42:37.444792 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051c11e-8db0-4cb4-8d23-6f0a06eb38bf None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:42:37.457293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051c11e-8db0-4cb4-8d23-6f0a06eb38bf None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:42:37.461925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002e841-cfe0-4c0d-8d85-cb463a723066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:42:37.462608 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002e841-cfe0-4c0d-8d85-cb463a723066 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:37.462893 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002e841-cfe0-4c0d-8d85-cb463a723066 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:37.463170 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002e841-cfe0-4c0d-8d85-cb463a723066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:37.473910 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002e841-cfe0-4c0d-8d85-cb463a723066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:42:37.475340 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002e841-cfe0-4c0d-8d85-cb463a723066 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:42:37.486065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 732/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:42:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:42:39.773137 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be1fc6-467f-436d-b0dd-dba2ba654435 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:39.773807 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be1fc6-467f-436d-b0dd-dba2ba654435 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:39.773807 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be1fc6-467f-436d-b0dd-dba2ba654435 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:39.775332 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 759/1491] 10.222.0.20 () {54 vars in 926 bytes} [Mon Jun 27 15:42:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:42:39.781433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8edd1-fc88-4491-9958-b213b785e49f None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:39.781992 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8edd1-fc88-4491-9958-b213b785e49f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:39.782111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8edd1-fc88-4491-9958-b213b785e49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:39.800676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-a5e8edd1-fc88-4491-9958-b213b785e49f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:42:39.801916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a5e8edd1-fc88-4491-9958-b213b785e49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:42:39.809752 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 733/1492] 10.222.0.20 () {58 vars in 1023 bytes} [Mon Jun 27 15:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:42:39.851729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be1fc6-467f-436d-b0dd-dba2ba654435 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:42:39.861392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be1fc6-467f-436d-b0dd-dba2ba654435 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:42:39.865037 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544489c-565d-4846-add8-19e3564aa892 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:42:39.865705 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544489c-565d-4846-add8-19e3564aa892 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:42:39.865988 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544489c-565d-4846-add8-19e3564aa892 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:42:39.866259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544489c-565d-4846-add8-19e3564aa892 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:42:39.876563 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544489c-565d-4846-add8-19e3564aa892 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:42:39.878073 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544489c-565d-4846-add8-19e3564aa892 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:42:39.888053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 760/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:42:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:42:49.795872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=33 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:42:49.883616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=33 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:44:10.172426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74cb3d-e241-4433-ada6-700ee91b9f62 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:44:10.173058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74cb3d-e241-4433-ada6-700ee91b9f62 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:44:10.173058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74cb3d-e241-4433-ada6-700ee91b9f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:44:10.193699 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-3f74cb3d-e241-4433-ada6-700ee91b9f62 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:44:10.194924 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3f74cb3d-e241-4433-ada6-700ee91b9f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:44:10.203957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 734/1494] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:44:10.218376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4136f08-c07a-4ff1-92be-7ad58780a50f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:44:10.218937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4136f08-c07a-4ff1-92be-7ad58780a50f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:44:10.218937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4136f08-c07a-4ff1-92be-7ad58780a50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:44:10.238677 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-e4136f08-c07a-4ff1-92be-7ad58780a50f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:44:10.239914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e4136f08-c07a-4ff1-92be-7ad58780a50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:44:10.249147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 761/1495] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:44:10.254409 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74cb3d-e241-4433-ada6-700ee91b9f62 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:44:10.265769 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74cb3d-e241-4433-ada6-700ee91b9f62 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:44:10.271073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb465b6-c52b-4fbe-9d1a-a711f3d72228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:44:10.271984 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb465b6-c52b-4fbe-9d1a-a711f3d72228 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:44:10.272454 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb465b6-c52b-4fbe-9d1a-a711f3d72228 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:44:10.272776 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb465b6-c52b-4fbe-9d1a-a711f3d72228 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:44:10.273665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb465b6-c52b-4fbe-9d1a-a711f3d72228 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:44:10.274051 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb465b6-c52b-4fbe-9d1a-a711f3d72228 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:44:10.275317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: WARNING py.warnings [None req-cbb465b6-c52b-4fbe-9d1a-a711f3d72228 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:44:10.275317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: warnings.warn(msg) Jun 27 15:44:10.275317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: Jun 27 15:44:10.276215 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb465b6-c52b-4fbe-9d1a-a711f3d72228 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:44:10.291155 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 735/1496] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:44:10.300322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4136f08-c07a-4ff1-92be-7ad58780a50f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:44:10.309983 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4136f08-c07a-4ff1-92be-7ad58780a50f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:44:10.315150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a78e57-e69d-4e53-b8d4-ee3465ac75aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:44:10.315824 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a78e57-e69d-4e53-b8d4-ee3465ac75aa admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:44:10.316106 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a78e57-e69d-4e53-b8d4-ee3465ac75aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:44:10.316382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a78e57-e69d-4e53-b8d4-ee3465ac75aa admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:44:10.317094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a78e57-e69d-4e53-b8d4-ee3465ac75aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:44:10.318958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: WARNING py.warnings [None req-f6a78e57-e69d-4e53-b8d4-ee3465ac75aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:44:10.318958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: warnings.warn(msg) Jun 27 15:44:10.318958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: Jun 27 15:44:10.320174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a78e57-e69d-4e53-b8d4-ee3465ac75aa admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:44:10.373533 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 762/1497] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:44:10 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:44:10.375151 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb465b6-c52b-4fbe-9d1a-a711f3d72228 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:44:10.386796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb465b6-c52b-4fbe-9d1a-a711f3d72228 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:44:10.458023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d9b91-3128-46d7-b342-dac14672bf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:44:10.459056 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d9b91-3128-46d7-b342-dac14672bf9d admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:44:10.459432 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d9b91-3128-46d7-b342-dac14672bf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:44:10.459727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d9b91-3128-46d7-b342-dac14672bf9d admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:44:10.460704 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d9b91-3128-46d7-b342-dac14672bf9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:44:10.462852 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: WARNING py.warnings [None req-6a6d9b91-3128-46d7-b342-dac14672bf9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:44:10.462852 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: warnings.warn(msg) Jun 27 15:44:10.462852 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: Jun 27 15:44:10.464094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d9b91-3128-46d7-b342-dac14672bf9d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:44:10.488620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 736/1498] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:44:10 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:44:10.491340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a78e57-e69d-4e53-b8d4-ee3465ac75aa admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:44:10.502649 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a78e57-e69d-4e53-b8d4-ee3465ac75aa admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:44:10.508328 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7951ed-1d07-4c2f-8d7e-bf3827e3224c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:44:10.509041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7951ed-1d07-4c2f-8d7e-bf3827e3224c admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:44:10.509343 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7951ed-1d07-4c2f-8d7e-bf3827e3224c admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:44:10.509670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7951ed-1d07-4c2f-8d7e-bf3827e3224c admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:44:10.510507 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7951ed-1d07-4c2f-8d7e-bf3827e3224c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:44:10.511523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: WARNING py.warnings [None req-db7951ed-1d07-4c2f-8d7e-bf3827e3224c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:44:10.511523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: warnings.warn(msg) Jun 27 15:44:10.511523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: Jun 27 15:44:10.512457 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7951ed-1d07-4c2f-8d7e-bf3827e3224c admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:44:10.527218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 763/1499] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:44:10.532149 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d9b91-3128-46d7-b342-dac14672bf9d admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:44:10.543476 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d9b91-3128-46d7-b342-dac14672bf9d admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:44:10.549236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e8149-7b0d-4987-a251-6737fdc2ace9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:44:10.549985 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e8149-7b0d-4987-a251-6737fdc2ace9 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:44:10.550265 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e8149-7b0d-4987-a251-6737fdc2ace9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:44:10.550539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e8149-7b0d-4987-a251-6737fdc2ace9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:44:10.551280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e8149-7b0d-4987-a251-6737fdc2ace9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:44:10.552316 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: WARNING py.warnings [None req-da6e8149-7b0d-4987-a251-6737fdc2ace9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:44:10.552316 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: warnings.warn(msg) Jun 27 15:44:10.552316 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: Jun 27 15:44:10.553229 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e8149-7b0d-4987-a251-6737fdc2ace9 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:44:10.568765 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 737/1500] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:44:10.573248 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7951ed-1d07-4c2f-8d7e-bf3827e3224c admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:44:10.584020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7951ed-1d07-4c2f-8d7e-bf3827e3224c admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:44:10.588774 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338274f-b068-4b81-abf8-2462c160691b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:44:10.589705 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338274f-b068-4b81-abf8-2462c160691b admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:44:10.590012 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338274f-b068-4b81-abf8-2462c160691b admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:44:10.590272 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338274f-b068-4b81-abf8-2462c160691b admin admin] PATH_INFO: `/v3/projects/b105cb2fd1d645438cfacc4ea2fef910/users/498adea9a659425f8c58bfe773eaaba8/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:44:10.606127 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338274f-b068-4b81-abf8-2462c160691b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b105cb2fd1d645438cfacc4ea2fef910, user_id=498adea9a659425f8c58bfe773eaaba8, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:44:10.607675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: WARNING py.warnings [None req-5338274f-b068-4b81-abf8-2462c160691b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:44:10.607675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: warnings.warn(msg) Jun 27 15:44:10.607675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: Jun 27 15:44:10.609696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338274f-b068-4b81-abf8-2462c160691b admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:44:10.626399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 764/1501] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:44:10 2022] PUT /identity/v3/projects/b105cb2fd1d645438cfacc4ea2fef910/users/498adea9a659425f8c58bfe773eaaba8/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:44:10.639489 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434dc17e-d3ad-4146-9a89-62158bf77117 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:44:10.639780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434dc17e-d3ad-4146-9a89-62158bf77117 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:44:10.640066 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434dc17e-d3ad-4146-9a89-62158bf77117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:44:10.669965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-434dc17e-d3ad-4146-9a89-62158bf77117 None None] MFA Rules not processed for user `498adea9a659425f8c58bfe773eaaba8`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:44:10.671507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-434dc17e-d3ad-4146-9a89-62158bf77117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:44:10.751260 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 738/1502] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:44:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:44:10.766341 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09081efb-23dd-4a90-a2b0-9ab8de1814a2 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:44:10.766629 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09081efb-23dd-4a90-a2b0-9ab8de1814a2 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:44:10.766911 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09081efb-23dd-4a90-a2b0-9ab8de1814a2 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:44:10.768309 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 765/1503] 10.222.0.20 () {54 vars in 963 bytes} [Mon Jun 27 15:44:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:44:10.773834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53e9fc-7f7d-426e-bf74-9a5eff6c435a None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:44:10.774135 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53e9fc-7f7d-426e-bf74-9a5eff6c435a None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:44:10.774498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53e9fc-7f7d-426e-bf74-9a5eff6c435a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:44:10.798704 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-cc53e9fc-7f7d-426e-bf74-9a5eff6c435a None None] MFA Rules not processed for user `51404afd35b94c47ba5c44f0deb73e30`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:44:10.800043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-cc53e9fc-7f7d-426e-bf74-9a5eff6c435a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:44:10.858181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:44:10.869753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 739/1504] 10.222.0.20 () {58 vars in 1060 bytes} [Mon Jun 27 15:44:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:44:10.881484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09081efb-23dd-4a90-a2b0-9ab8de1814a2 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:44:10.894971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09081efb-23dd-4a90-a2b0-9ab8de1814a2 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:44:10.898819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f81cf9a-66e5-4089-a2c6-75f8afea8be9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:44:10.899492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81cf9a-66e5-4089-a2c6-75f8afea8be9 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:44:10.899781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81cf9a-66e5-4089-a2c6-75f8afea8be9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:44:10.900061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81cf9a-66e5-4089-a2c6-75f8afea8be9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:44:10.910252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f81cf9a-66e5-4089-a2c6-75f8afea8be9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:44:10.911751 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f81cf9a-66e5-4089-a2c6-75f8afea8be9 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:44:10.923220 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 766/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:44:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:44:10.973955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459944cf-e1ad-4c7f-80df-601c7b072d25 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:44:10.974244 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459944cf-e1ad-4c7f-80df-601c7b072d25 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:44:10.974526 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459944cf-e1ad-4c7f-80df-601c7b072d25 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:44:10.975901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 740/1506] 10.222.0.20 () {52 vars in 829 bytes} [Mon Jun 27 15:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:44:10.980317 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f81cf9a-66e5-4089-a2c6-75f8afea8be9 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:44:11.057734 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f81cf9a-66e5-4089-a2c6-75f8afea8be9 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:44:11.064019 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58de76-24a0-4298-9f18-a2211f584c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:44:11.065129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58de76-24a0-4298-9f18-a2211f584c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:44:11.065411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58de76-24a0-4298-9f18-a2211f584c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:44:11.065727 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58de76-24a0-4298-9f18-a2211f584c3a admin admin] PATH_INFO: `/v3/users/498adea9a659425f8c58bfe773eaaba8` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:44:11.066953 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58de76-24a0-4298-9f18-a2211f584c3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=498adea9a659425f8c58bfe773eaaba8)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:44:11.068106 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: WARNING py.warnings [None req-bf58de76-24a0-4298-9f18-a2211f584c3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:44:11.068106 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: warnings.warn(msg) Jun 27 15:44:11.068106 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: Jun 27 15:44:11.069611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58de76-24a0-4298-9f18-a2211f584c3a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:44:11.140834 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-bf58de76-24a0-4298-9f18-a2211f584c3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '498adea9a659425f8c58bfe773eaaba8'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:44:11.161685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-bf58de76-24a0-4298-9f18-a2211f584c3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '498adea9a659425f8c58bfe773eaaba8'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:44:11.162458 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-bf58de76-24a0-4298-9f18-a2211f584c3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '498adea9a659425f8c58bfe773eaaba8'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:44:11.172852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-bf58de76-24a0-4298-9f18-a2211f584c3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '498adea9a659425f8c58bfe773eaaba8'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:44:11.182599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:44:11.189442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-bf58de76-24a0-4298-9f18-a2211f584c3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '498adea9a659425f8c58bfe773eaaba8'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:44:11.198675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 767/1507] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:44:10 2022] DELETE /identity/v3/users/498adea9a659425f8c58bfe773eaaba8 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:44:11.204950 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb315e7f-869c-44ce-821e-d7d8d52a1d88 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:44:11.205480 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb315e7f-869c-44ce-821e-d7d8d52a1d88 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:44:11.205837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb315e7f-869c-44ce-821e-d7d8d52a1d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:44:11.225838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-eb315e7f-869c-44ce-821e-d7d8d52a1d88 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:44:11.227059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-eb315e7f-869c-44ce-821e-d7d8d52a1d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:44:11.288857 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 741/1508] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:44:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:44:11.298962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58de76-24a0-4298-9f18-a2211f584c3a admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:44:11.300452 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-bf58de76-24a0-4298-9f18-a2211f584c3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:44:11.361175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58de76-24a0-4298-9f18-a2211f584c3a admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:44:11.364947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a633e7-8cf7-4f65-96fe-f02e1afec59b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:44:11.365608 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a633e7-8cf7-4f65-96fe-f02e1afec59b service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:44:11.365923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a633e7-8cf7-4f65-96fe-f02e1afec59b service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:44:11.366225 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a633e7-8cf7-4f65-96fe-f02e1afec59b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:44:11.376937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a633e7-8cf7-4f65-96fe-f02e1afec59b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:44:11.378423 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a633e7-8cf7-4f65-96fe-f02e1afec59b service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:44:11.388182 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 768/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:44:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:44:11.776793 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb315e7f-869c-44ce-821e-d7d8d52a1d88 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:44:11.778442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-eb315e7f-869c-44ce-821e-d7d8d52a1d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:44:11.792672 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb315e7f-869c-44ce-821e-d7d8d52a1d88 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:44:11.798531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be42ecf-4ff1-4777-9d43-ddd8b0083bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:44:11.800021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be42ecf-4ff1-4777-9d43-ddd8b0083bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:44:11.800534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be42ecf-4ff1-4777-9d43-ddd8b0083bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:44:11.801045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be42ecf-4ff1-4777-9d43-ddd8b0083bd4 admin admin] PATH_INFO: `/v3/projects/b105cb2fd1d645438cfacc4ea2fef910` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:44:11.803293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be42ecf-4ff1-4777-9d43-ddd8b0083bd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b105cb2fd1d645438cfacc4ea2fef910)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:44:11.805182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: WARNING py.warnings [None req-3be42ecf-4ff1-4777-9d43-ddd8b0083bd4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:44:11.805182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: warnings.warn(msg) Jun 27 15:44:11.805182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: Jun 27 15:44:11.807205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be42ecf-4ff1-4777-9d43-ddd8b0083bd4 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:44:11.885930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3be42ecf-4ff1-4777-9d43-ddd8b0083bd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b105cb2fd1d645438cfacc4ea2fef910', 'request_id': 'req-3be42ecf-4ff1-4777-9d43-ddd8b0083bd4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:44:11.900570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3be42ecf-4ff1-4777-9d43-ddd8b0083bd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b105cb2fd1d645438cfacc4ea2fef910', 'request_id': 'req-3be42ecf-4ff1-4777-9d43-ddd8b0083bd4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:44:11.907442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3be42ecf-4ff1-4777-9d43-ddd8b0083bd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b105cb2fd1d645438cfacc4ea2fef910', 'request_id': 'req-3be42ecf-4ff1-4777-9d43-ddd8b0083bd4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:44:11.923517 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3be42ecf-4ff1-4777-9d43-ddd8b0083bd4 admin admin] The token cache is being invalidate because project b105cb2fd1d645438cfacc4ea2fef910 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:44:11.923724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3be42ecf-4ff1-4777-9d43-ddd8b0083bd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:44:11.925651 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3be42ecf-4ff1-4777-9d43-ddd8b0083bd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:44:11.927265 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 742/1510] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:44:11 2022] DELETE /identity/v3/projects/b105cb2fd1d645438cfacc4ea2fef910 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:44:21.383822 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=29 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:44:21.919859 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:45:52.883895 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8224de0-f1bf-4de2-b203-1e8b502a2284 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:45:52.884530 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8224de0-f1bf-4de2-b203-1e8b502a2284 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:45:52.884530 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8224de0-f1bf-4de2-b203-1e8b502a2284 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:45:52.886039 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 769/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:45:52.891010 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdd79c-3235-4b00-8831-bd51fe93d8f3 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:45:52.891507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdd79c-3235-4b00-8831-bd51fe93d8f3 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:45:52.891624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdd79c-3235-4b00-8831-bd51fe93d8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:45:52.914528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-d0bdd79c-3235-4b00-8831-bd51fe93d8f3 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:45:52.915752 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d0bdd79c-3235-4b00-8831-bd51fe93d8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:45:52.980238 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 743/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:45:53.134986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8224de0-f1bf-4de2-b203-1e8b502a2284 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:45:53.137894 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e8224de0-f1bf-4de2-b203-1e8b502a2284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:45:53.212860 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8224de0-f1bf-4de2-b203-1e8b502a2284 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:45:53.218426 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d96490-9124-4c55-864f-931a4d4a85a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:45:53.219006 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d96490-9124-4c55-864f-931a4d4a85a2 service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:45:53.219292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d96490-9124-4c55-864f-931a4d4a85a2 service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:45:53.219570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d96490-9124-4c55-864f-931a4d4a85a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:45:53.227918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d96490-9124-4c55-864f-931a4d4a85a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:45:53.230250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d96490-9124-4c55-864f-931a4d4a85a2 service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:45:53.239706 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 770/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:45:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:45:54.072382 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f1657-c563-4cc9-9901-86c8133efbbe None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:45:54.073450 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f1657-c563-4cc9-9901-86c8133efbbe None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:45:54.073450 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f1657-c563-4cc9-9901-86c8133efbbe None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:45:54.074958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 744/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:45:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:45:54.079941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ce2b8-c1a9-4192-bef1-b83d087385f5 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:45:54.080680 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ce2b8-c1a9-4192-bef1-b83d087385f5 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:45:54.080680 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ce2b8-c1a9-4192-bef1-b83d087385f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:45:54.099313 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-700ce2b8-c1a9-4192-bef1-b83d087385f5 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:45:54.100530 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-700ce2b8-c1a9-4192-bef1-b83d087385f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:45:54.109251 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 771/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:45:54.262749 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f1657-c563-4cc9-9901-86c8133efbbe None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:45:54.277847 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f1657-c563-4cc9-9901-86c8133efbbe None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:45:54.283563 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4914da8-e5e7-4d49-a705-ac69b7e6b705 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:45:54.284255 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4914da8-e5e7-4d49-a705-ac69b7e6b705 service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:45:54.284542 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4914da8-e5e7-4d49-a705-ac69b7e6b705 service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:45:54.284823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4914da8-e5e7-4d49-a705-ac69b7e6b705 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:45:54.293829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4914da8-e5e7-4d49-a705-ac69b7e6b705 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:45:54.295498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4914da8-e5e7-4d49-a705-ac69b7e6b705 service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:45:54.305469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 745/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:45:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:45:54.446676 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a0c1b-9e09-47bf-a940-483c5f7ca4c9 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:45:54.446966 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a0c1b-9e09-47bf-a940-483c5f7ca4c9 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:45:54.447278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a0c1b-9e09-47bf-a940-483c5f7ca4c9 None None] PATH_INFO: `/v3` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:45:54.449337 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 772/1517] 10.222.0.48 () {54 vars in 925 bytes} [Mon Jun 27 15:45:54 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:45:54.455772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e825d-f3a0-443e-b6d1-93308158fa35 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:45:54.456065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e825d-f3a0-443e-b6d1-93308158fa35 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:45:54.456346 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e825d-f3a0-443e-b6d1-93308158fa35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:45:54.474845 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-303e825d-f3a0-443e-b6d1-93308158fa35 None None] MFA Rules not processed for user `ffafb68f1bb941588a5217ad83ad907c`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:45:54.476055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-303e825d-f3a0-443e-b6d1-93308158fa35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:45:54.543160 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 746/1518] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Jun 27 15:45:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:46:03.513330 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaece7f7-27e2-41f1-8e56-fc029624d14e None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:03.513903 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaece7f7-27e2-41f1-8e56-fc029624d14e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:03.513903 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaece7f7-27e2-41f1-8e56-fc029624d14e None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:03.515210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 773/1519] 10.222.0.20 () {54 vars in 960 bytes} [Mon Jun 27 15:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:46:03.520297 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718b9e6-f728-41ad-a122-00380ccf6e8b None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:03.520762 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718b9e6-f728-41ad-a122-00380ccf6e8b None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:03.520877 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718b9e6-f728-41ad-a122-00380ccf6e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:03.539443 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-a718b9e6-f728-41ad-a122-00380ccf6e8b None None] MFA Rules not processed for user `ffafb68f1bb941588a5217ad83ad907c`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:46:03.540668 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a718b9e6-f728-41ad-a122-00380ccf6e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:46:03.550749 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 747/1520] 10.222.0.20 () {58 vars in 1057 bytes} [Mon Jun 27 15:46:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:46:03.558470 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaece7f7-27e2-41f1-8e56-fc029624d14e None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:46:03.571635 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaece7f7-27e2-41f1-8e56-fc029624d14e None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:46:03.576527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ce371-bb2d-4db3-b55f-92e9ccf921db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffafb68f1bb941588a5217ad83ad907c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:46:03.577192 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ce371-bb2d-4db3-b55f-92e9ccf921db service placement] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:03.577473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ce371-bb2d-4db3-b55f-92e9ccf921db service placement] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:03.577854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ce371-bb2d-4db3-b55f-92e9ccf921db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:03.586541 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ce371-bb2d-4db3-b55f-92e9ccf921db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:46:03.588206 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ce371-bb2d-4db3-b55f-92e9ccf921db service placement] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:46:03.598740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 774/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:46:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:46:05.604930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb354c9f-e95d-4a8e-911a-6e665e54b5f8 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:05.605533 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb354c9f-e95d-4a8e-911a-6e665e54b5f8 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:05.605664 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb354c9f-e95d-4a8e-911a-6e665e54b5f8 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:05.607073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 748/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:46:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:46:05.612046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d965d-8ecd-4196-80f9-d5ac77ef75ce None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:05.612458 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d965d-8ecd-4196-80f9-d5ac77ef75ce None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:05.612614 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d965d-8ecd-4196-80f9-d5ac77ef75ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:05.631341 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-0a1d965d-8ecd-4196-80f9-d5ac77ef75ce None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:46:05.632557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0a1d965d-8ecd-4196-80f9-d5ac77ef75ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:46:05.640965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 775/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:46:05.807561 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb354c9f-e95d-4a8e-911a-6e665e54b5f8 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:46:05.822227 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb354c9f-e95d-4a8e-911a-6e665e54b5f8 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:46:05.827644 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e708d9-7035-48eb-ae4d-f63af0257c2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:46:05.828292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e708d9-7035-48eb-ae4d-f63af0257c2e service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:05.828574 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e708d9-7035-48eb-ae4d-f63af0257c2e service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:05.828847 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e708d9-7035-48eb-ae4d-f63af0257c2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:05.837627 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e708d9-7035-48eb-ae4d-f63af0257c2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:46:05.839779 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e708d9-7035-48eb-ae4d-f63af0257c2e service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:46:05.850140 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 749/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:46:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:46:06.690046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53490166-bf0a-48ef-9a9e-5913c5132556 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:06.690624 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53490166-bf0a-48ef-9a9e-5913c5132556 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:06.690624 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53490166-bf0a-48ef-9a9e-5913c5132556 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:06.692055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 776/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 15:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:46:06.696877 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d592f6-e8ce-4782-81bf-6da26b210641 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:06.697307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d592f6-e8ce-4782-81bf-6da26b210641 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:06.697438 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d592f6-e8ce-4782-81bf-6da26b210641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:06.718339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-a8d592f6-e8ce-4782-81bf-6da26b210641 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:46:06.719569 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a8d592f6-e8ce-4782-81bf-6da26b210641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:46:06.727478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 750/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 15:46:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:46:06.880513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53490166-bf0a-48ef-9a9e-5913c5132556 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:46:06.895061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53490166-bf0a-48ef-9a9e-5913c5132556 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:46:06.900494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27277e17-dd9a-41c5-be9c-c7bc61833f46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:46:06.901200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27277e17-dd9a-41c5-be9c-c7bc61833f46 service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:06.901539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27277e17-dd9a-41c5-be9c-c7bc61833f46 service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:06.901816 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27277e17-dd9a-41c5-be9c-c7bc61833f46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:06.910515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27277e17-dd9a-41c5-be9c-c7bc61833f46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:46:06.912113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27277e17-dd9a-41c5-be9c-c7bc61833f46 service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:46:06.922716 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 777/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:46:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:46:10.913977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d60c12-c8b8-4dd6-91b8-15dbfc47f299 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:10.914584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d60c12-c8b8-4dd6-91b8-15dbfc47f299 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:10.914584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d60c12-c8b8-4dd6-91b8-15dbfc47f299 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:10.915948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 751/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:46:10.921695 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027238be-c84a-4ad6-a62a-0cbf1d6a1bce None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:10.922190 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027238be-c84a-4ad6-a62a-0cbf1d6a1bce None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:10.922296 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027238be-c84a-4ad6-a62a-0cbf1d6a1bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:10.940683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-027238be-c84a-4ad6-a62a-0cbf1d6a1bce None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:46:10.941915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-027238be-c84a-4ad6-a62a-0cbf1d6a1bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:46:10.950295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 778/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:46:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:46:10.987624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d60c12-c8b8-4dd6-91b8-15dbfc47f299 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:46:10.989501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-08d60c12-c8b8-4dd6-91b8-15dbfc47f299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:46:11.052272 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d60c12-c8b8-4dd6-91b8-15dbfc47f299 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:46:11.055909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f77f4-046b-45fc-8892-f82637b48be4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:46:11.056562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f77f4-046b-45fc-8892-f82637b48be4 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:11.056855 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f77f4-046b-45fc-8892-f82637b48be4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:11.057859 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f77f4-046b-45fc-8892-f82637b48be4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:11.067325 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f77f4-046b-45fc-8892-f82637b48be4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:46:11.068763 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f77f4-046b-45fc-8892-f82637b48be4 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:46:11.074450 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:46:11.079683 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 752/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:46:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:46:13.228665 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0eb461-e691-41bb-b33c-57a3f3e2874f None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:13.230152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0eb461-e691-41bb-b33c-57a3f3e2874f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:13.230152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0eb461-e691-41bb-b33c-57a3f3e2874f None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:13.231389 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 779/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:46:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:46:13.236760 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe0247-fa3f-4d27-a3e6-2c09e005f3fb None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:13.237221 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe0247-fa3f-4d27-a3e6-2c09e005f3fb None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:13.237331 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe0247-fa3f-4d27-a3e6-2c09e005f3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:13.256325 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-32fe0247-fa3f-4d27-a3e6-2c09e005f3fb None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:46:13.257540 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-32fe0247-fa3f-4d27-a3e6-2c09e005f3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:46:13.266121 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 753/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:46:13.305023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0eb461-e691-41bb-b33c-57a3f3e2874f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:46:13.317154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0eb461-e691-41bb-b33c-57a3f3e2874f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:46:13.320701 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e447c-e711-4fbf-a3ba-020acf4f6a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:46:13.321341 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e447c-e711-4fbf-a3ba-020acf4f6a3d service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:13.321627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e447c-e711-4fbf-a3ba-020acf4f6a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:13.321937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e447c-e711-4fbf-a3ba-020acf4f6a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:13.333028 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e447c-e711-4fbf-a3ba-020acf4f6a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:46:13.334508 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e447c-e711-4fbf-a3ba-020acf4f6a3d service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:46:13.348790 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 780/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:46:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:46:16.820922 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fc6b0-6bd2-4e08-b4f5-8f95687f8e26 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:16.821561 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fc6b0-6bd2-4e08-b4f5-8f95687f8e26 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:16.821561 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fc6b0-6bd2-4e08-b4f5-8f95687f8e26 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:16.823651 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 754/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:46:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:46:16.829222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cea445-c556-49f3-bbee-a89efcd5c0fa None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:16.829681 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cea445-c556-49f3-bbee-a89efcd5c0fa None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:16.829836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cea445-c556-49f3-bbee-a89efcd5c0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:16.848826 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-e0cea445-c556-49f3-bbee-a89efcd5c0fa None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:46:16.850066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e0cea445-c556-49f3-bbee-a89efcd5c0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:46:16.858513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 781/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:46:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:46:17.171056 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fc6b0-6bd2-4e08-b4f5-8f95687f8e26 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:46:17.172793 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d23fc6b0-6bd2-4e08-b4f5-8f95687f8e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:46:17.235684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fc6b0-6bd2-4e08-b4f5-8f95687f8e26 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:46:17.239247 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e16c92-d5d1-449c-b9f3-a146a342af51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:46:17.239901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e16c92-d5d1-449c-b9f3-a146a342af51 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:17.240185 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e16c92-d5d1-449c-b9f3-a146a342af51 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:17.240466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e16c92-d5d1-449c-b9f3-a146a342af51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:17.250085 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e16c92-d5d1-449c-b9f3-a146a342af51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:46:17.251527 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e16c92-d5d1-449c-b9f3-a146a342af51 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:46:17.261780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 755/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:46:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:46:19.379887 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35984c1d-5232-4468-81b7-ea489616bc95 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:19.380512 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35984c1d-5232-4468-81b7-ea489616bc95 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:19.380512 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35984c1d-5232-4468-81b7-ea489616bc95 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:19.382204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 782/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 15:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:46:19.388872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70231cb0-0190-4ab8-9078-9c95b39f9dca None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:19.389727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70231cb0-0190-4ab8-9078-9c95b39f9dca None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:19.389727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70231cb0-0190-4ab8-9078-9c95b39f9dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:19.409856 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-70231cb0-0190-4ab8-9078-9c95b39f9dca None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:46:19.411098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-70231cb0-0190-4ab8-9078-9c95b39f9dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:46:19.419870 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 756/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 15:46:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:46:19.734061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35984c1d-5232-4468-81b7-ea489616bc95 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:46:19.735815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-35984c1d-5232-4468-81b7-ea489616bc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:46:19.750585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35984c1d-5232-4468-81b7-ea489616bc95 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:46:19.754204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3394019-9e4a-41fe-922e-af2ec020e7c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:46:19.754869 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3394019-9e4a-41fe-922e-af2ec020e7c8 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:46:19.755158 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3394019-9e4a-41fe-922e-af2ec020e7c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:46:19.755439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3394019-9e4a-41fe-922e-af2ec020e7c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:46:19.765118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3394019-9e4a-41fe-922e-af2ec020e7c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:46:19.766633 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3394019-9e4a-41fe-922e-af2ec020e7c8 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:46:19.776463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 783/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:46:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:46:29.404115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=32 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:46:29.772206 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=70 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:49.095084 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccc5d2-13ad-479f-9bfe-b065222ce189 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.095748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccc5d2-13ad-479f-9bfe-b065222ce189 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.095748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccc5d2-13ad-479f-9bfe-b065222ce189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.122883 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-38ccc5d2-13ad-479f-9bfe-b065222ce189 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:49.124199 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-38ccc5d2-13ad-479f-9bfe-b065222ce189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:49.133719 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 757/1540] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:49.143368 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b07c48-77e3-4879-a7ea-3dc9f87acf9f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.143972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b07c48-77e3-4879-a7ea-3dc9f87acf9f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.143972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b07c48-77e3-4879-a7ea-3dc9f87acf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.178562 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-22b07c48-77e3-4879-a7ea-3dc9f87acf9f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:49.180801 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-22b07c48-77e3-4879-a7ea-3dc9f87acf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:49.193492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 784/1541] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:49.199371 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ccc5d2-13ad-479f-9bfe-b065222ce189 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:49.211703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ccc5d2-13ad-479f-9bfe-b065222ce189 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:49.216854 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697c542-cd91-473e-ae46-ed158da320c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:49.217551 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697c542-cd91-473e-ae46-ed158da320c0 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.217874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697c542-cd91-473e-ae46-ed158da320c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.218159 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697c542-cd91-473e-ae46-ed158da320c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.218914 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697c542-cd91-473e-ae46-ed158da320c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:47:49.219226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697c542-cd91-473e-ae46-ed158da320c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:49.220867 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697c542-cd91-473e-ae46-ed158da320c0 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:49.250462 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 758/1542] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:47:49.260239 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b07c48-77e3-4879-a7ea-3dc9f87acf9f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:49.274829 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b07c48-77e3-4879-a7ea-3dc9f87acf9f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:49.282054 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb869f-ca64-4f1a-a741-f8a77f8228aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:49.282755 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb869f-ca64-4f1a-a741-f8a77f8228aa admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.283043 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb869f-ca64-4f1a-a741-f8a77f8228aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.283324 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb869f-ca64-4f1a-a741-f8a77f8228aa admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.284103 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb869f-ca64-4f1a-a741-f8a77f8228aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:49.286172 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb869f-ca64-4f1a-a741-f8a77f8228aa admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:49.326318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 785/1543] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:47:49 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:47:49.330644 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697c542-cd91-473e-ae46-ed158da320c0 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:49.343276 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a0c2f-5987-4b32-968c-e920c8bc999f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.344828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a0c2f-5987-4b32-968c-e920c8bc999f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.345169 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a0c2f-5987-4b32-968c-e920c8bc999f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.352634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697c542-cd91-473e-ae46-ed158da320c0 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:49.393554 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-817a0c2f-5987-4b32-968c-e920c8bc999f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:49.394848 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-817a0c2f-5987-4b32-968c-e920c8bc999f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:49.532148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b1354-05a9-493a-9848-d2e13ebc90a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:49.533064 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b1354-05a9-493a-9848-d2e13ebc90a8 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.533389 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b1354-05a9-493a-9848-d2e13ebc90a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.533902 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b1354-05a9-493a-9848-d2e13ebc90a8 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.535112 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b1354-05a9-493a-9848-d2e13ebc90a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:49.537682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b1354-05a9-493a-9848-d2e13ebc90a8 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:49.573210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 786/1544] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:47:49.574447 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55d435-f4e9-473a-9437-c784f8e8546e None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.574736 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55d435-f4e9-473a-9437-c784f8e8546e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.575014 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55d435-f4e9-473a-9437-c784f8e8546e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.577389 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 759/1545] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:47:49 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:49.579072 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0c636-daf4-4a36-b01a-9dacd0690298 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.579372 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0c636-daf4-4a36-b01a-9dacd0690298 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.579657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0c636-daf4-4a36-b01a-9dacd0690298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.615477 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-fc55d435-f4e9-473a-9437-c784f8e8546e None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:49.616664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-fc55d435-f4e9-473a-9437-c784f8e8546e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:49.618302 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-b7c0c636-daf4-4a36-b01a-9dacd0690298 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:49.619875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-b7c0c636-daf4-4a36-b01a-9dacd0690298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:49.627739 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 787/1546] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:49.628388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55d435-f4e9-473a-9437-c784f8e8546e None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:49.630640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 760/1547] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:49.632282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce224f1-7088-46c9-9d12-a9113e14cade None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.633675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce224f1-7088-46c9-9d12-a9113e14cade None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.634202 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce224f1-7088-46c9-9d12-a9113e14cade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.639566 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55d435-f4e9-473a-9437-c784f8e8546e None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:49.645200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af5c0b-f1f3-42a5-b443-cfd3ab0707f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:49.645931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af5c0b-f1f3-42a5-b443-cfd3ab0707f4 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.646215 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af5c0b-f1f3-42a5-b443-cfd3ab0707f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.646490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af5c0b-f1f3-42a5-b443-cfd3ab0707f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.647226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af5c0b-f1f3-42a5-b443-cfd3ab0707f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:49.649548 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af5c0b-f1f3-42a5-b443-cfd3ab0707f4 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:49.658545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-fce224f1-7088-46c9-9d12-a9113e14cade None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:49.659769 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-fce224f1-7088-46c9-9d12-a9113e14cade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:49.664908 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 788/1548] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:49.667052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cbda6-b60b-48a8-94f8-d8d45c21eb59 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.667696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cbda6-b60b-48a8-94f8-d8d45c21eb59 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.668258 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cbda6-b60b-48a8-94f8-d8d45c21eb59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.673066 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 761/1549] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:49.674596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62c22a-01e6-410e-8b4f-7fb6089afdcf None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.674957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62c22a-01e6-410e-8b4f-7fb6089afdcf None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.675322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62c22a-01e6-410e-8b4f-7fb6089afdcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.692311 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-dc0cbda6-b60b-48a8-94f8-d8d45c21eb59 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:49.693907 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-dc0cbda6-b60b-48a8-94f8-d8d45c21eb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:49.703082 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-5d62c22a-01e6-410e-8b4f-7fb6089afdcf None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:49.703437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 789/1550] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:49.704710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596cf39-4eb1-41c1-8ab1-b843b3aae00b None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.705000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596cf39-4eb1-41c1-8ab1-b843b3aae00b None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.705277 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b596cf39-4eb1-41c1-8ab1-b843b3aae00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.707233 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-5d62c22a-01e6-410e-8b4f-7fb6089afdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:49.717541 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 762/1551] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:49.718890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6942cb-9a9a-4993-9848-263715c07a35 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.719187 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6942cb-9a9a-4993-9848-263715c07a35 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.719472 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6942cb-9a9a-4993-9848-263715c07a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.727064 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-b596cf39-4eb1-41c1-8ab1-b843b3aae00b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:49.728418 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-b596cf39-4eb1-41c1-8ab1-b843b3aae00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:49.740705 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-bc6942cb-9a9a-4993-9848-263715c07a35 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:49.741027 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 790/1552] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:49.742561 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-bc6942cb-9a9a-4993-9848-263715c07a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:49.742725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596cf39-4eb1-41c1-8ab1-b843b3aae00b None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:49.751744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 763/1553] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:49.752571 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6942cb-9a9a-4993-9848-263715c07a35 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:49.754552 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b596cf39-4eb1-41c1-8ab1-b843b3aae00b None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:49.759589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9ddeb-7c05-4938-8cf1-762e3548ebf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:49.760310 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9ddeb-7c05-4938-8cf1-762e3548ebf1 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.760600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9ddeb-7c05-4938-8cf1-762e3548ebf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.760880 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9ddeb-7c05-4938-8cf1-762e3548ebf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.761619 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9ddeb-7c05-4938-8cf1-762e3548ebf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:49.762863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6942cb-9a9a-4993-9848-263715c07a35 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:49.763316 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9ddeb-7c05-4938-8cf1-762e3548ebf1 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:49.767992 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90340d93-139e-4846-9154-4c0b98e56a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:49.768714 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90340d93-139e-4846-9154-4c0b98e56a68 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.768945 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90340d93-139e-4846-9154-4c0b98e56a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.769218 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90340d93-139e-4846-9154-4c0b98e56a68 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.769976 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90340d93-139e-4846-9154-4c0b98e56a68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:47:49.770286 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90340d93-139e-4846-9154-4c0b98e56a68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:49.771907 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90340d93-139e-4846-9154-4c0b98e56a68 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:49.775044 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 791/1554] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:49.776376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b0a12-0837-495a-98cf-bc808fc91c4b None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.776699 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b0a12-0837-495a-98cf-bc808fc91c4b None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.776968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b0a12-0837-495a-98cf-bc808fc91c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.786796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 764/1555] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:47:49.792730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943c03f-e18e-4f55-9637-da0e468772e5 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.793024 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943c03f-e18e-4f55-9637-da0e468772e5 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.793392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943c03f-e18e-4f55-9637-da0e468772e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.797801 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-b17b0a12-0837-495a-98cf-bc808fc91c4b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:49.799022 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-b17b0a12-0837-495a-98cf-bc808fc91c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:49.808490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 792/1556] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:49.809823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd159895-9f61-4227-acdb-6ffa18ecaf94 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.810270 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd159895-9f61-4227-acdb-6ffa18ecaf94 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.810439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd159895-9f61-4227-acdb-6ffa18ecaf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.813002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-9943c03f-e18e-4f55-9637-da0e468772e5 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:49.814258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-9943c03f-e18e-4f55-9637-da0e468772e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:49.822962 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 765/1557] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:49.824073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ae1c3-e5d2-4c00-9434-ad0cc321449b None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.824363 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ae1c3-e5d2-4c00-9434-ad0cc321449b None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.824647 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ae1c3-e5d2-4c00-9434-ad0cc321449b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.833416 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-bd159895-9f61-4227-acdb-6ffa18ecaf94 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:49.834652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-bd159895-9f61-4227-acdb-6ffa18ecaf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:49.843516 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 793/1558] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:49.843962 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-da8ae1c3-e5d2-4c00-9434-ad0cc321449b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:49.844437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd159895-9f61-4227-acdb-6ffa18ecaf94 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:49.845197 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-da8ae1c3-e5d2-4c00-9434-ad0cc321449b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:49.855066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd159895-9f61-4227-acdb-6ffa18ecaf94 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:49.857734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 766/1559] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:49.858579 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ae1c3-e5d2-4c00-9434-ad0cc321449b None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:49.861407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426fb20-a3d6-48f0-87b6-186e86930405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:49.862120 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426fb20-a3d6-48f0-87b6-186e86930405 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.862407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426fb20-a3d6-48f0-87b6-186e86930405 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.862681 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426fb20-a3d6-48f0-87b6-186e86930405 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.863413 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426fb20-a3d6-48f0-87b6-186e86930405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:47:49.863721 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426fb20-a3d6-48f0-87b6-186e86930405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:49.864731 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: WARNING py.warnings [None req-b426fb20-a3d6-48f0-87b6-186e86930405 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:47:49.864731 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: warnings.warn(msg) Jun 27 15:47:49.864731 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: Jun 27 15:47:49.865942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426fb20-a3d6-48f0-87b6-186e86930405 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:49.871877 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ae1c3-e5d2-4c00-9434-ad0cc321449b None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:49.876690 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cce62-3fce-43e0-9ba2-c6f614716788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:49.877269 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cce62-3fce-43e0-9ba2-c6f614716788 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.877555 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cce62-3fce-43e0-9ba2-c6f614716788 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.877897 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cce62-3fce-43e0-9ba2-c6f614716788 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.878597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cce62-3fce-43e0-9ba2-c6f614716788 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:47:49.878912 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cce62-3fce-43e0-9ba2-c6f614716788 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:49.880100 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 794/1560] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:47:49.880531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cce62-3fce-43e0-9ba2-c6f614716788 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:49.880725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426fb20-a3d6-48f0-87b6-186e86930405 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:49.892884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426fb20-a3d6-48f0-87b6-186e86930405 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:49.895947 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 767/1561] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:47:49.898682 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d380d-c45a-4a5e-a289-c90c99e2609c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:49.899940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cce62-3fce-43e0-9ba2-c6f614716788 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:49.900263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d380d-c45a-4a5e-a289-c90c99e2609c admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.900585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d380d-c45a-4a5e-a289-c90c99e2609c admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.900825 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d380d-c45a-4a5e-a289-c90c99e2609c admin admin] PATH_INFO: `/v3/projects/2a805612e76c4f5e9052aa4a8ead24f1/users/3520bd4387a140f88558d5f6fe55986e/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.912214 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cce62-3fce-43e0-9ba2-c6f614716788 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:49.917396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d380d-c45a-4a5e-a289-c90c99e2609c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a805612e76c4f5e9052aa4a8ead24f1, user_id=3520bd4387a140f88558d5f6fe55986e, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:49.918501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be81910-9928-4e98-a778-a48da3eecba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:49.919178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be81910-9928-4e98-a778-a48da3eecba9 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.919469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be81910-9928-4e98-a778-a48da3eecba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.919759 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be81910-9928-4e98-a778-a48da3eecba9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.920120 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d380d-c45a-4a5e-a289-c90c99e2609c admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:49.920433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be81910-9928-4e98-a778-a48da3eecba9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:49.921792 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: WARNING py.warnings [None req-1be81910-9928-4e98-a778-a48da3eecba9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:47:49.921792 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: warnings.warn(msg) Jun 27 15:47:49.921792 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: Jun 27 15:47:49.923027 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be81910-9928-4e98-a778-a48da3eecba9 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:49.934682 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 795/1562] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:47:49 2022] PUT /identity/v3/projects/2a805612e76c4f5e9052aa4a8ead24f1/users/3520bd4387a140f88558d5f6fe55986e/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:47:49.935901 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418aea75-0d62-4c6f-b565-a4ba129f1dee None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:49.936241 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418aea75-0d62-4c6f-b565-a4ba129f1dee None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:49.936453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418aea75-0d62-4c6f-b565-a4ba129f1dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:49.954275 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 768/1563] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:47:49 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:49.954699 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be81910-9928-4e98-a778-a48da3eecba9 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:49.957917 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-418aea75-0d62-4c6f-b565-a4ba129f1dee None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:49.959118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-418aea75-0d62-4c6f-b565-a4ba129f1dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:49.996344 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:50.004935 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:50.024155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 796/1564] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:50.028480 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be81910-9928-4e98-a778-a48da3eecba9 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.028668 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5935ea6-ba77-4130-80ac-863baff0464d None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.028668 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5935ea6-ba77-4130-80ac-863baff0464d None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.028668 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5935ea6-ba77-4130-80ac-863baff0464d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.030605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842ac45-df2f-4f2f-850f-3fa5cc9edb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.031308 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842ac45-df2f-4f2f-850f-3fa5cc9edb86 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.031593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842ac45-df2f-4f2f-850f-3fa5cc9edb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.031871 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842ac45-df2f-4f2f-850f-3fa5cc9edb86 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.032650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842ac45-df2f-4f2f-850f-3fa5cc9edb86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:47:50.032964 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842ac45-df2f-4f2f-850f-3fa5cc9edb86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.034661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842ac45-df2f-4f2f-850f-3fa5cc9edb86 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.046333 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-f5935ea6-ba77-4130-80ac-863baff0464d None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:50.047538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f5935ea6-ba77-4130-80ac-863baff0464d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:50.056067 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 769/1565] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:47:50.056067 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842ac45-df2f-4f2f-850f-3fa5cc9edb86 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.057050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 797/1566] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:50.057898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5935ea6-ba77-4130-80ac-863baff0464d None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.060728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842ac45-df2f-4f2f-850f-3fa5cc9edb86 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.066811 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57572d5-8d9d-46c1-9735-10b84d24ea19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.067502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57572d5-8d9d-46c1-9735-10b84d24ea19 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.067820 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57572d5-8d9d-46c1-9735-10b84d24ea19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.068101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57572d5-8d9d-46c1-9735-10b84d24ea19 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.068214 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5935ea6-ba77-4130-80ac-863baff0464d None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.068843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57572d5-8d9d-46c1-9735-10b84d24ea19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:47:50.069190 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57572d5-8d9d-46c1-9735-10b84d24ea19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.071107 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57572d5-8d9d-46c1-9735-10b84d24ea19 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.073217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f832efbe-cf72-4051-97b6-848f70bedf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.073954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832efbe-cf72-4051-97b6-848f70bedf9a admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.074237 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832efbe-cf72-4051-97b6-848f70bedf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.074524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832efbe-cf72-4051-97b6-848f70bedf9a admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.075215 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f832efbe-cf72-4051-97b6-848f70bedf9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.077673 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f832efbe-cf72-4051-97b6-848f70bedf9a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.084457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 770/1567] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:47:50.085307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57572d5-8d9d-46c1-9735-10b84d24ea19 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.095879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57572d5-8d9d-46c1-9735-10b84d24ea19 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.100668 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035832c-4cf4-4180-995e-c41ec3b7fe72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.101432 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035832c-4cf4-4180-995e-c41ec3b7fe72 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.101747 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035832c-4cf4-4180-995e-c41ec3b7fe72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.102187 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035832c-4cf4-4180-995e-c41ec3b7fe72 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.102938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035832c-4cf4-4180-995e-c41ec3b7fe72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.105451 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035832c-4cf4-4180-995e-c41ec3b7fe72 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.108718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 798/1568] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:47:50 2022] POST /identity/v3/projects => generated 355 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:50.110187 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831d067-3563-4ce1-9369-dbd5b5337598 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.110526 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831d067-3563-4ce1-9369-dbd5b5337598 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.110842 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831d067-3563-4ce1-9369-dbd5b5337598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.132540 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-1831d067-3563-4ce1-9369-dbd5b5337598 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:50.133710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1831d067-3563-4ce1-9369-dbd5b5337598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:50.137930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 771/1569] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:47:50 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:50.138682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035832c-4cf4-4180-995e-c41ec3b7fe72 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.201796 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 799/1570] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:50.205686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831d067-3563-4ce1-9369-dbd5b5337598 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.212428 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035832c-4cf4-4180-995e-c41ec3b7fe72 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.212687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831d067-3563-4ce1-9369-dbd5b5337598 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.217062 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163dd8f-1661-44b2-adf9-b33c2d044913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.217211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f04aa-7747-46cd-be07-2ae8944ce403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.217783 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163dd8f-1661-44b2-adf9-b33c2d044913 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.217921 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f04aa-7747-46cd-be07-2ae8944ce403 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.218072 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163dd8f-1661-44b2-adf9-b33c2d044913 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.218200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f04aa-7747-46cd-be07-2ae8944ce403 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.218354 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163dd8f-1661-44b2-adf9-b33c2d044913 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.218481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f04aa-7747-46cd-be07-2ae8944ce403 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.219049 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163dd8f-1661-44b2-adf9-b33c2d044913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.219249 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f04aa-7747-46cd-be07-2ae8944ce403 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:47:50.219562 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f04aa-7747-46cd-be07-2ae8944ce403 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.221001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163dd8f-1661-44b2-adf9-b33c2d044913 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.221345 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f04aa-7747-46cd-be07-2ae8944ce403 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.237240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 800/1571] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:47:50.238291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f04aa-7747-46cd-be07-2ae8944ce403 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.249249 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f04aa-7747-46cd-be07-2ae8944ce403 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.256104 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 772/1572] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:47:50 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:50.256104 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163dd8f-1661-44b2-adf9-b33c2d044913 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.256372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b83c7-e07d-4c19-b57f-0ceafce07fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.256372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b83c7-e07d-4c19-b57f-0ceafce07fac admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.256372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b83c7-e07d-4c19-b57f-0ceafce07fac admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.256372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b83c7-e07d-4c19-b57f-0ceafce07fac admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.256665 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b83c7-e07d-4c19-b57f-0ceafce07fac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.259583 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b83c7-e07d-4c19-b57f-0ceafce07fac admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.264914 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163dd8f-1661-44b2-adf9-b33c2d044913 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.269628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93947165-d144-45a6-aa0d-be9732242a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.270320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93947165-d144-45a6-aa0d-be9732242a51 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.270634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93947165-d144-45a6-aa0d-be9732242a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.270913 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93947165-d144-45a6-aa0d-be9732242a51 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.271639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93947165-d144-45a6-aa0d-be9732242a51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:47:50.271947 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93947165-d144-45a6-aa0d-be9732242a51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.274229 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93947165-d144-45a6-aa0d-be9732242a51 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.287459 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 773/1573] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:47:50.288430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93947165-d144-45a6-aa0d-be9732242a51 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.289777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 801/1574] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:47:50 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:50.289777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b83c7-e07d-4c19-b57f-0ceafce07fac admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.306983 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93947165-d144-45a6-aa0d-be9732242a51 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.360968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b83c7-e07d-4c19-b57f-0ceafce07fac admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.366083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d192d-b90a-49d3-b8fa-d36c2b87ec0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.366769 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d192d-b90a-49d3-b8fa-d36c2b87ec0f admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.367096 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d192d-b90a-49d3-b8fa-d36c2b87ec0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.367640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d192d-b90a-49d3-b8fa-d36c2b87ec0f admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.368657 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d192d-b90a-49d3-b8fa-d36c2b87ec0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.369410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c64a93-5887-4579-8df7-d66dffc43031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.370204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c64a93-5887-4579-8df7-d66dffc43031 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.370491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: WARNING py.warnings [None req-c77d192d-b90a-49d3-b8fa-d36c2b87ec0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:47:50.370491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: warnings.warn(msg) Jun 27 15:47:50.370491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: Jun 27 15:47:50.370712 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c64a93-5887-4579-8df7-d66dffc43031 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.370795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c64a93-5887-4579-8df7-d66dffc43031 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.371613 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c64a93-5887-4579-8df7-d66dffc43031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.372790 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77d192d-b90a-49d3-b8fa-d36c2b87ec0f admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.374092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c64a93-5887-4579-8df7-d66dffc43031 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.396854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 802/1575] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:47:50 2022] POST /identity/v3/users => generated 278 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:50.398699 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d192d-b90a-49d3-b8fa-d36c2b87ec0f admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.410780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 774/1576] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:47:50 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:50.410780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c64a93-5887-4579-8df7-d66dffc43031 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.411089 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d192d-b90a-49d3-b8fa-d36c2b87ec0f admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.413682 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:50.415164 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:50.475514 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c64a93-5887-4579-8df7-d66dffc43031 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.485748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5d9f7-0169-4497-8a5b-de61d7dd0db1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.486777 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5d9f7-0169-4497-8a5b-de61d7dd0db1 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.487124 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5d9f7-0169-4497-8a5b-de61d7dd0db1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.487478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5d9f7-0169-4497-8a5b-de61d7dd0db1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.562348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5d9f7-0169-4497-8a5b-de61d7dd0db1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.564330 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5d9f7-0169-4497-8a5b-de61d7dd0db1 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.576836 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 775/1577] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:47:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:47:50.577764 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5d9f7-0169-4497-8a5b-de61d7dd0db1 service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.593008 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5d9f7-0169-4497-8a5b-de61d7dd0db1 service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.597960 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce2db6-19ce-40e8-969c-3101ca50048d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.598648 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce2db6-19ce-40e8-969c-3101ca50048d admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.598966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce2db6-19ce-40e8-969c-3101ca50048d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.599547 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce2db6-19ce-40e8-969c-3101ca50048d admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.600091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce2db6-19ce-40e8-969c-3101ca50048d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.602678 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce2db6-19ce-40e8-969c-3101ca50048d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.607146 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f8245-8e94-4d0f-9434-6b6619bf749f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.607907 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f8245-8e94-4d0f-9434-6b6619bf749f admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.608193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f8245-8e94-4d0f-9434-6b6619bf749f admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.608501 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f8245-8e94-4d0f-9434-6b6619bf749f admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.609232 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f8245-8e94-4d0f-9434-6b6619bf749f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.611247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f8245-8e94-4d0f-9434-6b6619bf749f admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.638711 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 803/1578] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:47:50 2022] POST /identity/v3/users => generated 292 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:50.643431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f8245-8e94-4d0f-9434-6b6619bf749f admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.643995 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 776/1579] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:47:50 2022] POST /identity/v3/projects => generated 391 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:50.645451 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce2db6-19ce-40e8-969c-3101ca50048d admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.665439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f8245-8e94-4d0f-9434-6b6619bf749f admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.734474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce2db6-19ce-40e8-969c-3101ca50048d admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.739773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223534e9-3009-4ed9-853e-cbfea1861583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.740602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223534e9-3009-4ed9-853e-cbfea1861583 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.741004 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223534e9-3009-4ed9-853e-cbfea1861583 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.741374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223534e9-3009-4ed9-853e-cbfea1861583 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.742231 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223534e9-3009-4ed9-853e-cbfea1861583 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.744365 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223534e9-3009-4ed9-853e-cbfea1861583 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.764965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b6c35-422d-41eb-8803-91ca53b05922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.766182 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b6c35-422d-41eb-8803-91ca53b05922 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.766869 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b6c35-422d-41eb-8803-91ca53b05922 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.767322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 777/1580] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:47:50 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:50.768517 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223534e9-3009-4ed9-853e-cbfea1861583 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.770017 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b6c35-422d-41eb-8803-91ca53b05922 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.771316 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b6c35-422d-41eb-8803-91ca53b05922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.774199 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b6c35-422d-41eb-8803-91ca53b05922 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.780267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223534e9-3009-4ed9-853e-cbfea1861583 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.786065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47088f9a-eebc-4ef1-a0df-8685de437ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.786982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47088f9a-eebc-4ef1-a0df-8685de437ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.787266 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47088f9a-eebc-4ef1-a0df-8685de437ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.788369 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47088f9a-eebc-4ef1-a0df-8685de437ac9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.788369 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47088f9a-eebc-4ef1-a0df-8685de437ac9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.789767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 804/1581] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:50.790993 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47088f9a-eebc-4ef1-a0df-8685de437ac9 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.793541 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b6c35-422d-41eb-8803-91ca53b05922 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.812024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b6c35-422d-41eb-8803-91ca53b05922 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.819061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138b88c-72c1-410f-9031-0f68e91baecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.819690 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138b88c-72c1-410f-9031-0f68e91baecb admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.820118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138b88c-72c1-410f-9031-0f68e91baecb admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.825378 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138b88c-72c1-410f-9031-0f68e91baecb admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.825378 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138b88c-72c1-410f-9031-0f68e91baecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.825378 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138b88c-72c1-410f-9031-0f68e91baecb admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.831950 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:50.838618 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 805/1582] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:50.839457 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138b88c-72c1-410f-9031-0f68e91baecb admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.851299 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138b88c-72c1-410f-9031-0f68e91baecb admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.858750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9e89a-6a28-4c80-b53d-83cbea2a4fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.859634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9e89a-6a28-4c80-b53d-83cbea2a4fdc admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.860074 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9e89a-6a28-4c80-b53d-83cbea2a4fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.860501 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9e89a-6a28-4c80-b53d-83cbea2a4fdc admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.861785 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9e89a-6a28-4c80-b53d-83cbea2a4fdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.864354 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 778/1583] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:47:50 2022] POST /identity/v3/projects => generated 401 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:50.865354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9e89a-6a28-4c80-b53d-83cbea2a4fdc admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.865474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47088f9a-eebc-4ef1-a0df-8685de437ac9 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.895272 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 806/1584] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:47:50 2022] POST /identity/v3/users => generated 283 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:50.896201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9e89a-6a28-4c80-b53d-83cbea2a4fdc admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:50.908500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9e89a-6a28-4c80-b53d-83cbea2a4fdc admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.966697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47088f9a-eebc-4ef1-a0df-8685de437ac9 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:50.971833 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9dc75d-d859-42c5-83d6-31ca9428a8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:50.972519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9dc75d-d859-42c5-83d6-31ca9428a8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:50.972806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9dc75d-d859-42c5-83d6-31ca9428a8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:50.973087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9dc75d-d859-42c5-83d6-31ca9428a8e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:50.974630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9dc75d-d859-42c5-83d6-31ca9428a8e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:50.976645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9dc75d-d859-42c5-83d6-31ca9428a8e2 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:50.994997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 779/1585] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:50.994997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9dc75d-d859-42c5-83d6-31ca9428a8e2 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:51.006538 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9dc75d-d859-42c5-83d6-31ca9428a8e2 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:51.008215 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e10c5b-be03-4e0a-b250-25dfabdc0fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:51.009409 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e10c5b-be03-4e0a-b250-25dfabdc0fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.010101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e10c5b-be03-4e0a-b250-25dfabdc0fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.010600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e10c5b-be03-4e0a-b250-25dfabdc0fd0 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.012000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e10c5b-be03-4e0a-b250-25dfabdc0fd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:51.012154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05e235-0f16-4a35-8298-ee25554526f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:51.013220 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05e235-0f16-4a35-8298-ee25554526f9 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.013582 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05e235-0f16-4a35-8298-ee25554526f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.013947 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05e235-0f16-4a35-8298-ee25554526f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.015316 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05e235-0f16-4a35-8298-ee25554526f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:51.016191 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e10c5b-be03-4e0a-b250-25dfabdc0fd0 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:51.022923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05e235-0f16-4a35-8298-ee25554526f9 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:51.030284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:51.051321 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 780/1586] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:51.052434 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05e235-0f16-4a35-8298-ee25554526f9 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:51.061848 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 807/1587] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:47:50 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:51.061848 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e10c5b-be03-4e0a-b250-25dfabdc0fd0 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:51.064421 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05e235-0f16-4a35-8298-ee25554526f9 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:51.070831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64f6a3-3d47-4715-97b7-fb8adcc75a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:51.071079 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64f6a3-3d47-4715-97b7-fb8adcc75a70 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.071409 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64f6a3-3d47-4715-97b7-fb8adcc75a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.071882 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64f6a3-3d47-4715-97b7-fb8adcc75a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.072596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64f6a3-3d47-4715-97b7-fb8adcc75a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:51.074778 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64f6a3-3d47-4715-97b7-fb8adcc75a70 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:51.079338 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e10c5b-be03-4e0a-b250-25dfabdc0fd0 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:51.090603 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc56aa-7e32-4dfd-a38e-d2f33d7e4f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:51.090879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 781/1588] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:51.091821 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64f6a3-3d47-4715-97b7-fb8adcc75a70 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:51.096612 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc56aa-7e32-4dfd-a38e-d2f33d7e4f8e admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.097107 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc56aa-7e32-4dfd-a38e-d2f33d7e4f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.097788 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc56aa-7e32-4dfd-a38e-d2f33d7e4f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.099060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc56aa-7e32-4dfd-a38e-d2f33d7e4f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:51.102850 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc56aa-7e32-4dfd-a38e-d2f33d7e4f8e admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:51.104149 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64f6a3-3d47-4715-97b7-fb8adcc75a70 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:51.109958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21ee91-d442-4e28-9401-92b999d09b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:51.110688 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21ee91-d442-4e28-9401-92b999d09b35 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.110974 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21ee91-d442-4e28-9401-92b999d09b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.111252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21ee91-d442-4e28-9401-92b999d09b35 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.112246 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21ee91-d442-4e28-9401-92b999d09b35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:51.114743 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21ee91-d442-4e28-9401-92b999d09b35 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:51.132799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 808/1589] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:51.132799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc56aa-7e32-4dfd-a38e-d2f33d7e4f8e admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:51.147648 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 782/1590] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:47:51 2022] POST /identity/v3/users => generated 301 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:51.149172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21ee91-d442-4e28-9401-92b999d09b35 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:51.157939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc56aa-7e32-4dfd-a38e-d2f33d7e4f8e admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:51.162027 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21ee91-d442-4e28-9401-92b999d09b35 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:51.167948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f57a8-13b5-4225-932a-e3fc6e3b6600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:51.168641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f57a8-13b5-4225-932a-e3fc6e3b6600 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.168930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f57a8-13b5-4225-932a-e3fc6e3b6600 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.169206 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f57a8-13b5-4225-932a-e3fc6e3b6600 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.169565 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34945d-0abf-45bf-8129-c084dcc43c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:51.169978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f57a8-13b5-4225-932a-e3fc6e3b6600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:51.170747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34945d-0abf-45bf-8129-c084dcc43c73 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.171511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34945d-0abf-45bf-8129-c084dcc43c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.172007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34945d-0abf-45bf-8129-c084dcc43c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.172133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f57a8-13b5-4225-932a-e3fc6e3b6600 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:51.173413 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34945d-0abf-45bf-8129-c084dcc43c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:51.176634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34945d-0abf-45bf-8129-c084dcc43c73 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:51.187932 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 783/1591] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:51.187932 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f57a8-13b5-4225-932a-e3fc6e3b6600 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:51.202223 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 809/1592] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:51.202223 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34945d-0abf-45bf-8129-c084dcc43c73 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:51.202831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f57a8-13b5-4225-932a-e3fc6e3b6600 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:51.207989 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4e0bb-3300-44d7-a3ff-ef6482dcac2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:51.208738 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4e0bb-3300-44d7-a3ff-ef6482dcac2e admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.209101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4e0bb-3300-44d7-a3ff-ef6482dcac2e admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.209457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4e0bb-3300-44d7-a3ff-ef6482dcac2e admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.210322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4e0bb-3300-44d7-a3ff-ef6482dcac2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:51.212577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4e0bb-3300-44d7-a3ff-ef6482dcac2e admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:51.220845 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34945d-0abf-45bf-8129-c084dcc43c73 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:51.225749 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 784/1593] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:51.227118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4e0bb-3300-44d7-a3ff-ef6482dcac2e admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:51.234883 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723e784-136e-4be0-8277-6d93fb1199a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:51.235671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723e784-136e-4be0-8277-6d93fb1199a7 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.236106 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723e784-136e-4be0-8277-6d93fb1199a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.236446 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723e784-136e-4be0-8277-6d93fb1199a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.237383 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723e784-136e-4be0-8277-6d93fb1199a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:51.239337 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723e784-136e-4be0-8277-6d93fb1199a7 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:51.239687 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4e0bb-3300-44d7-a3ff-ef6482dcac2e admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:51.246195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4d702-f99b-4357-ab26-878e605ba66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:51.247372 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4d702-f99b-4357-ab26-878e605ba66b admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.248305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4d702-f99b-4357-ab26-878e605ba66b admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.248305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4d702-f99b-4357-ab26-878e605ba66b admin admin] PATH_INFO: `/v3/projects/5b5846565bf74c5cac3ff5c3168cfd48/users/d35c652539f443d4895b5f81461833ba/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.260270 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 810/1594] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:51.260270 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723e784-136e-4be0-8277-6d93fb1199a7 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:51.271760 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723e784-136e-4be0-8277-6d93fb1199a7 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:51.277321 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4d702-f99b-4357-ab26-878e605ba66b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b5846565bf74c5cac3ff5c3168cfd48, user_id=d35c652539f443d4895b5f81461833ba, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:51.277582 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347c1fb-3d90-4b86-b812-90d6fd388a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:51.278552 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347c1fb-3d90-4b86-b812-90d6fd388a8a admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.278701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: WARNING py.warnings [None req-06d4d702-f99b-4357-ab26-878e605ba66b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:47:51.278701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: warnings.warn(msg) Jun 27 15:47:51.278701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: Jun 27 15:47:51.279179 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347c1fb-3d90-4b86-b812-90d6fd388a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.279532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347c1fb-3d90-4b86-b812-90d6fd388a8a admin admin] PATH_INFO: `/v3/projects/6481f389f5524085b205c153f4ff8556/users/3513b60703944443ac2950859215eb2c/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.281206 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4d702-f99b-4357-ab26-878e605ba66b admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:51.301445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 785/1595] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:47:51 2022] PUT /identity/v3/projects/5b5846565bf74c5cac3ff5c3168cfd48/users/d35c652539f443d4895b5f81461833ba/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:47:51.301704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a347c1fb-3d90-4b86-b812-90d6fd388a8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6481f389f5524085b205c153f4ff8556, user_id=3513b60703944443ac2950859215eb2c, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:51.301825 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4d702-f99b-4357-ab26-878e605ba66b admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:51.301916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a347c1fb-3d90-4b86-b812-90d6fd388a8a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:51.319301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 811/1596] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:47:51 2022] PUT /identity/v3/projects/6481f389f5524085b205c153f4ff8556/users/3513b60703944443ac2950859215eb2c/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:47:51.321292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347c1fb-3d90-4b86-b812-90d6fd388a8a admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:51.379733 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4d702-f99b-4357-ab26-878e605ba66b admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:51.394901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:51.417184 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347c1fb-3d90-4b86-b812-90d6fd388a8a admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:51.423164 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c406dc-1d24-4974-b59e-bc9871844d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:51.424099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c406dc-1d24-4974-b59e-bc9871844d69 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.424387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c406dc-1d24-4974-b59e-bc9871844d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.424664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c406dc-1d24-4974-b59e-bc9871844d69 admin admin] PATH_INFO: `/v3/projects/4b12bd7199c94e6392710d44c8b8c24e/users/61c7ee6276174c5f88e86810f632e690/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.460436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c406dc-1d24-4974-b59e-bc9871844d69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b12bd7199c94e6392710d44c8b8c24e, user_id=61c7ee6276174c5f88e86810f632e690, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:51.464601 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903d30a-3295-4e40-8550-f89271c44a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:51.465296 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903d30a-3295-4e40-8550-f89271c44a29 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.465571 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903d30a-3295-4e40-8550-f89271c44a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.465832 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c406dc-1d24-4974-b59e-bc9871844d69 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:51.466012 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903d30a-3295-4e40-8550-f89271c44a29 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.466833 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7903d30a-3295-4e40-8550-f89271c44a29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:51.468828 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7903d30a-3295-4e40-8550-f89271c44a29 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:51.491790 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 812/1597] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:47:51 2022] PUT /identity/v3/projects/4b12bd7199c94e6392710d44c8b8c24e/users/61c7ee6276174c5f88e86810f632e690/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:47:51.492958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 786/1598] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:51.495851 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c406dc-1d24-4974-b59e-bc9871844d69 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:51.496055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903d30a-3295-4e40-8550-f89271c44a29 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:51.513895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903d30a-3295-4e40-8550-f89271c44a29 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:51.633555 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d162f-f5e3-4b7e-a750-be23dd6a1e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:51.634833 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d162f-f5e3-4b7e-a750-be23dd6a1e67 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.635633 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d162f-f5e3-4b7e-a750-be23dd6a1e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.635633 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d162f-f5e3-4b7e-a750-be23dd6a1e67 admin admin] PATH_INFO: `/v3/projects/b5557a813d8f41f9be8eac38ebf7886e/users/3e90e6a1993e408cb05d7072c5555d73/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.645522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c406dc-1d24-4974-b59e-bc9871844d69 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:51.653503 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62529019-ab8a-4691-b17e-eb5403e7c291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:51.654203 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62529019-ab8a-4691-b17e-eb5403e7c291 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.654491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62529019-ab8a-4691-b17e-eb5403e7c291 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.654778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62529019-ab8a-4691-b17e-eb5403e7c291 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.655615 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d162f-f5e3-4b7e-a750-be23dd6a1e67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5557a813d8f41f9be8eac38ebf7886e, user_id=3e90e6a1993e408cb05d7072c5555d73, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:51.655738 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62529019-ab8a-4691-b17e-eb5403e7c291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:51.657171 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62529019-ab8a-4691-b17e-eb5403e7c291 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:51.657723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d162f-f5e3-4b7e-a750-be23dd6a1e67 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:51.676021 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 813/1599] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:51.676021 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62529019-ab8a-4691-b17e-eb5403e7c291 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:51.677300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 787/1600] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:47:51 2022] PUT /identity/v3/projects/b5557a813d8f41f9be8eac38ebf7886e/users/3e90e6a1993e408cb05d7072c5555d73/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:47:51.678672 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a3b71-7d42-4cc0-8d06-aac341dca588 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.678965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a3b71-7d42-4cc0-8d06-aac341dca588 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.679827 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a3b71-7d42-4cc0-8d06-aac341dca588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.688570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62529019-ab8a-4691-b17e-eb5403e7c291 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:51.703064 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:51.733713 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-a03a3b71-7d42-4cc0-8d06-aac341dca588 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:51.735435 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a03a3b71-7d42-4cc0-8d06-aac341dca588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:51.778436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631e30f-cd3a-4173-bd9c-9285e336f747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:51.779118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631e30f-cd3a-4173-bd9c-9285e336f747 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.779399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631e30f-cd3a-4173-bd9c-9285e336f747 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.779670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631e30f-cd3a-4173-bd9c-9285e336f747 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.780396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4631e30f-cd3a-4173-bd9c-9285e336f747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:51.782062 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4631e30f-cd3a-4173-bd9c-9285e336f747 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:51.795756 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 814/1601] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:51.807711 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99dc10-756a-4cca-a98d-29cff0c2f797 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.808047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99dc10-756a-4cca-a98d-29cff0c2f797 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.808275 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99dc10-756a-4cca-a98d-29cff0c2f797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.840511 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 788/1602] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:51.841562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a3b71-7d42-4cc0-8d06-aac341dca588 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:51.851937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-da99dc10-756a-4cca-a98d-29cff0c2f797 None None] MFA Rules not processed for user `3513b60703944443ac2950859215eb2c`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:51.853664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-da99dc10-756a-4cca-a98d-29cff0c2f797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:51.861366 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a3b71-7d42-4cc0-8d06-aac341dca588 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:51.868652 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334584db-8789-4245-a2ff-2473eaa49453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:51.869826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334584db-8789-4245-a2ff-2473eaa49453 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.870119 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334584db-8789-4245-a2ff-2473eaa49453 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.870401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334584db-8789-4245-a2ff-2473eaa49453 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.871164 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334584db-8789-4245-a2ff-2473eaa49453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:51.874597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334584db-8789-4245-a2ff-2473eaa49453 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:51.892556 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 789/1603] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:51.893068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d3b84-5d35-447c-b32e-06546e8b62cc None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.893368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d3b84-5d35-447c-b32e-06546e8b62cc None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.893695 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d3b84-5d35-447c-b32e-06546e8b62cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.938980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-111d3b84-5d35-447c-b32e-06546e8b62cc None None] MFA Rules not processed for user `61c7ee6276174c5f88e86810f632e690`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:51.941028 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-111d3b84-5d35-447c-b32e-06546e8b62cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:51.946428 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 815/1604] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:51 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:51.946428 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99dc10-756a-4cca-a98d-29cff0c2f797 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:51.956170 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99dc10-756a-4cca-a98d-29cff0c2f797 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:51.958624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:51.962968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbb610-dd60-4dd3-86de-768cdbececff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:51.963960 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbb610-dd60-4dd3-86de-768cdbececff admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:51.964244 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbb610-dd60-4dd3-86de-768cdbececff admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:51.966133 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbb610-dd60-4dd3-86de-768cdbececff admin admin] PATH_INFO: `/v3/projects/8305be07ffd7400bb67e5e9c52668a35/users/1efcd6a2ce9e4a2ea7db5f9c0e21bbaa/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:51.985655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbbb610-dd60-4dd3-86de-768cdbececff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8305be07ffd7400bb67e5e9c52668a35, user_id=1efcd6a2ce9e4a2ea7db5f9c0e21bbaa, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:51.990138 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbbb610-dd60-4dd3-86de-768cdbececff admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:52.019196 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 816/1605] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:47:51 2022] PUT /identity/v3/projects/8305be07ffd7400bb67e5e9c52668a35/users/1efcd6a2ce9e4a2ea7db5f9c0e21bbaa/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:47:52.019196 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f6d8c-06cb-42bf-8ff7-df2de31fa59d None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:52.019196 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f6d8c-06cb-42bf-8ff7-df2de31fa59d None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:52.019196 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f6d8c-06cb-42bf-8ff7-df2de31fa59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:52.044575 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-f07f6d8c-06cb-42bf-8ff7-df2de31fa59d None None] MFA Rules not processed for user `3e90e6a1993e408cb05d7072c5555d73`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:52.046456 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f07f6d8c-06cb-42bf-8ff7-df2de31fa59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:52.055948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 790/1606] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:52.057051 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d3b84-5d35-447c-b32e-06546e8b62cc None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:52.069195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d3b84-5d35-447c-b32e-06546e8b62cc None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:52.149808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1003d4-8ad2-440f-926b-cb5055662de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:52.150724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1003d4-8ad2-440f-926b-cb5055662de4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:52.151007 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1003d4-8ad2-440f-926b-cb5055662de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:52.151840 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1003d4-8ad2-440f-926b-cb5055662de4 admin admin] PATH_INFO: `/v3/projects/f5e448e724fa479ea693168592686391/users/1a7a81da8acf40849a9ea86e2542ae6a/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:52.170840 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1003d4-8ad2-440f-926b-cb5055662de4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5e448e724fa479ea693168592686391, user_id=1a7a81da8acf40849a9ea86e2542ae6a, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:52.174225 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1003d4-8ad2-440f-926b-cb5055662de4 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:52.197074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 791/1607] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:47:52 2022] PUT /identity/v3/projects/f5e448e724fa479ea693168592686391/users/1a7a81da8acf40849a9ea86e2542ae6a/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:47:52.197074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1003d4-8ad2-440f-926b-cb5055662de4 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:52.197074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-6b1003d4-8ad2-440f-926b-cb5055662de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:52.227074 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 817/1608] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:52.231703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f6d8c-06cb-42bf-8ff7-df2de31fa59d None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:52.249299 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f6d8c-06cb-42bf-8ff7-df2de31fa59d None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:52.255982 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:52.280341 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1003d4-8ad2-440f-926b-cb5055662de4 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:52.285318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d65b6-cccc-49cd-8a73-bb23dbacc6c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:52.286422 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d65b6-cccc-49cd-8a73-bb23dbacc6c4 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:52.286698 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d65b6-cccc-49cd-8a73-bb23dbacc6c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:52.287245 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d65b6-cccc-49cd-8a73-bb23dbacc6c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:52.330229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e9a3d-0284-4e16-a515-709872f760ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:52.331334 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e9a3d-0284-4e16-a515-709872f760ce admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:52.331755 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e9a3d-0284-4e16-a515-709872f760ce admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:52.332254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e9a3d-0284-4e16-a515-709872f760ce admin admin] PATH_INFO: `/v3/projects/bfe198915ace4d5c87125cbfbb9f7cba/users/c03dd300681e4929879f8e08fb100347/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:52.355669 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e9a3d-0284-4e16-a515-709872f760ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfe198915ace4d5c87125cbfbb9f7cba, user_id=c03dd300681e4929879f8e08fb100347, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:52.356003 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:52.359750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e9a3d-0284-4e16-a515-709872f760ce admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:52.375365 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d65b6-cccc-49cd-8a73-bb23dbacc6c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:52.377126 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d65b6-cccc-49cd-8a73-bb23dbacc6c4 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:52.383301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 818/1609] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:47:52 2022] PUT /identity/v3/projects/bfe198915ace4d5c87125cbfbb9f7cba/users/c03dd300681e4929879f8e08fb100347/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:47:52.384760 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e9a3d-0284-4e16-a515-709872f760ce admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:52.392387 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 792/1610] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:47:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:47:52.394195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eed1f4-4c86-4228-9955-5a87eb61887e None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:52.394594 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eed1f4-4c86-4228-9955-5a87eb61887e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:52.395044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eed1f4-4c86-4228-9955-5a87eb61887e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:52.436558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-50eed1f4-4c86-4228-9955-5a87eb61887e None None] MFA Rules not processed for user `1efcd6a2ce9e4a2ea7db5f9c0e21bbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:52.437978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-50eed1f4-4c86-4228-9955-5a87eb61887e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:52.545150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e9a3d-0284-4e16-a515-709872f760ce admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:52.552472 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 793/1611] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:52.556809 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f54b6d-d3af-41ae-a6e7-7a0b149a69f3 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:52.557314 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f54b6d-d3af-41ae-a6e7-7a0b149a69f3 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:52.558093 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f54b6d-d3af-41ae-a6e7-7a0b149a69f3 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:52.560668 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 794/1612] 10.222.0.20 () {54 vars in 961 bytes} [Mon Jun 27 15:47:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:47:52.562918 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eca88e-3ade-4ba7-986c-76bda8c0ead8 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:52.563410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eca88e-3ade-4ba7-986c-76bda8c0ead8 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:52.563846 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eca88e-3ade-4ba7-986c-76bda8c0ead8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:52.565478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d98c3a-51e3-4797-89bd-2cdc1b98d40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:52.567517 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d98c3a-51e3-4797-89bd-2cdc1b98d40d admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:52.567809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d98c3a-51e3-4797-89bd-2cdc1b98d40d admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:52.568093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d98c3a-51e3-4797-89bd-2cdc1b98d40d admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:52.572262 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d98c3a-51e3-4797-89bd-2cdc1b98d40d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:52.575385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d98c3a-51e3-4797-89bd-2cdc1b98d40d admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:52.592505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-e3eca88e-3ade-4ba7-986c-76bda8c0ead8 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:52.594404 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e3eca88e-3ade-4ba7-986c-76bda8c0ead8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:52.605673 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 795/1613] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:52.606964 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eca88e-3ade-4ba7-986c-76bda8c0ead8 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:52.623235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eca88e-3ade-4ba7-986c-76bda8c0ead8 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:52.627958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 819/1614] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:47:52 2022] POST /identity/v3/projects => generated 357 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:52.629719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791aec2f-1398-4e41-a207-420e09e67517 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:52.630020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791aec2f-1398-4e41-a207-420e09e67517 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:52.630413 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791aec2f-1398-4e41-a207-420e09e67517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:52.638882 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3968f5-6f36-4981-ae58-ed88d886764a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:52.639709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3968f5-6f36-4981-ae58-ed88d886764a admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:52.640022 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3968f5-6f36-4981-ae58-ed88d886764a admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:52.640304 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3968f5-6f36-4981-ae58-ed88d886764a admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:52.641873 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3968f5-6f36-4981-ae58-ed88d886764a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:52.644095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3968f5-6f36-4981-ae58-ed88d886764a admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:52.679448 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-791aec2f-1398-4e41-a207-420e09e67517 None None] MFA Rules not processed for user `c03dd300681e4929879f8e08fb100347`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:52.681154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-791aec2f-1398-4e41-a207-420e09e67517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:52.708668 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 796/1615] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:47:52 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:52.709955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3968f5-6f36-4981-ae58-ed88d886764a admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:52.841366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 820/1616] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:52.841366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9985f4d-0a90-4412-bc99-53bb84214817 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:52.841366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9985f4d-0a90-4412-bc99-53bb84214817 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:52.841366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9985f4d-0a90-4412-bc99-53bb84214817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:52.841366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:52.866082 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-f9985f4d-0a90-4412-bc99-53bb84214817 None None] MFA Rules not processed for user `1c8f6262cf24419daf803ae71aed24f5`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:52.868019 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f9985f4d-0a90-4412-bc99-53bb84214817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:52.874905 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3968f5-6f36-4981-ae58-ed88d886764a admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:52.886332 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1637bc-9c01-43f1-a1d9-ba0932dd6e31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:52.886601 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1637bc-9c01-43f1-a1d9-ba0932dd6e31 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:52.886824 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1637bc-9c01-43f1-a1d9-ba0932dd6e31 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:52.887025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1637bc-9c01-43f1-a1d9-ba0932dd6e31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:52.979355 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 821/1617] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Jun 27 15:47:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:52.980602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9985f4d-0a90-4412-bc99-53bb84214817 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:53.041941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1637bc-9c01-43f1-a1d9-ba0932dd6e31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:53.043399 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1637bc-9c01-43f1-a1d9-ba0932dd6e31 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:53.066977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 797/1618] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:47:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:47:53.069711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1637bc-9c01-43f1-a1d9-ba0932dd6e31 service cinder] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:53.080693 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:53.139639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9985f4d-0a90-4412-bc99-53bb84214817 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:53.144958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459d825-f022-4407-84a4-9af153fef13f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:53.146388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459d825-f022-4407-84a4-9af153fef13f service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:53.147733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459d825-f022-4407-84a4-9af153fef13f service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:53.148854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459d825-f022-4407-84a4-9af153fef13f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:53.181898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1637bc-9c01-43f1-a1d9-ba0932dd6e31 service cinder] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:53.195408 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb07c06-9108-44ea-a6a9-f8500e52340e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:53.196115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb07c06-9108-44ea-a6a9-f8500e52340e admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:53.196404 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb07c06-9108-44ea-a6a9-f8500e52340e admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:53.196837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb07c06-9108-44ea-a6a9-f8500e52340e admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:53.197616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb07c06-9108-44ea-a6a9-f8500e52340e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:53.199726 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb07c06-9108-44ea-a6a9-f8500e52340e admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:53.244809 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 798/1619] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:47:53 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:53.254409 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb07c06-9108-44ea-a6a9-f8500e52340e admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:53.277620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb07c06-9108-44ea-a6a9-f8500e52340e admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:53.284615 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459d825-f022-4407-84a4-9af153fef13f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:53.286130 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459d825-f022-4407-84a4-9af153fef13f service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:53.293519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a06ca-a98f-479e-860b-853458c4aa32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:53.294244 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a06ca-a98f-479e-860b-853458c4aa32 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:53.294531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a06ca-a98f-479e-860b-853458c4aa32 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:53.294812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a06ca-a98f-479e-860b-853458c4aa32 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:53.295521 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a06ca-a98f-479e-860b-853458c4aa32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:53.298504 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a06ca-a98f-479e-860b-853458c4aa32 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:53.314269 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 822/1620] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:47:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:47:53.319113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459d825-f022-4407-84a4-9af153fef13f service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:53.333749 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459d825-f022-4407-84a4-9af153fef13f service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:53.341344 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7643a0-ebcf-4087-89b3-3f3416f911e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:53.342896 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7643a0-ebcf-4087-89b3-3f3416f911e8 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:53.343190 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7643a0-ebcf-4087-89b3-3f3416f911e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:53.343472 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7643a0-ebcf-4087-89b3-3f3416f911e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:53.344336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7643a0-ebcf-4087-89b3-3f3416f911e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:53.346506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7643a0-ebcf-4087-89b3-3f3416f911e8 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:53.355754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 799/1621] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:47:53 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:53.355754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a06ca-a98f-479e-860b-853458c4aa32 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:53.384078 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a06ca-a98f-479e-860b-853458c4aa32 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:53.389851 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 823/1622] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:47:53 2022] POST /identity/v3/projects => generated 405 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:53.390791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7643a0-ebcf-4087-89b3-3f3416f911e8 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:53.400930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16911019-71e5-473a-9e85-78f3f61183ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:53.401801 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16911019-71e5-473a-9e85-78f3f61183ed service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:53.401957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16911019-71e5-473a-9e85-78f3f61183ed service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:53.402237 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16911019-71e5-473a-9e85-78f3f61183ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:53.428677 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:53.487266 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7643a0-ebcf-4087-89b3-3f3416f911e8 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:53.497719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8da90-89fd-433e-91e7-cdbad0214989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:53.498467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8da90-89fd-433e-91e7-cdbad0214989 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:53.498752 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8da90-89fd-433e-91e7-cdbad0214989 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:53.499029 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8da90-89fd-433e-91e7-cdbad0214989 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:53.500590 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f8da90-89fd-433e-91e7-cdbad0214989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:53.506374 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f8da90-89fd-433e-91e7-cdbad0214989 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:53.523871 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 824/1623] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:53.524869 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8da90-89fd-433e-91e7-cdbad0214989 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:53.552625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8da90-89fd-433e-91e7-cdbad0214989 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:53.575950 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c11500-38c8-43fe-acec-fd956a292567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:53.577164 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c11500-38c8-43fe-acec-fd956a292567 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:53.577379 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c11500-38c8-43fe-acec-fd956a292567 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:53.577956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c11500-38c8-43fe-acec-fd956a292567 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:53.580412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c11500-38c8-43fe-acec-fd956a292567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:53.586638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c11500-38c8-43fe-acec-fd956a292567 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:53.597076 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16911019-71e5-473a-9e85-78f3f61183ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:53.605043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16911019-71e5-473a-9e85-78f3f61183ed service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:53.618523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 825/1624] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:53.618523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c11500-38c8-43fe-acec-fd956a292567 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:53.639498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 800/1625] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:47:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:47:53.643108 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16911019-71e5-473a-9e85-78f3f61183ed service nova] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:53.656500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c11500-38c8-43fe-acec-fd956a292567 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:53.674353 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbea836-fbe1-4a99-a214-42943ffe0483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:53.674496 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16911019-71e5-473a-9e85-78f3f61183ed service nova] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:53.676521 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbea836-fbe1-4a99-a214-42943ffe0483 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:53.676843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbea836-fbe1-4a99-a214-42943ffe0483 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:53.677152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbea836-fbe1-4a99-a214-42943ffe0483 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:53.677857 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbea836-fbe1-4a99-a214-42943ffe0483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:53.686524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbea836-fbe1-4a99-a214-42943ffe0483 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:53.694249 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23859ea-8329-4672-a3d2-ce7df443e2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:53.696500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23859ea-8329-4672-a3d2-ce7df443e2da admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:53.703040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23859ea-8329-4672-a3d2-ce7df443e2da admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:53.703757 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23859ea-8329-4672-a3d2-ce7df443e2da admin admin] PATH_INFO: `/v3/projects/cd25c5eab42c4d319217c324cb13acb8/users/c97a085ef91e4c8392091477e6596c57/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:53.767210 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23859ea-8329-4672-a3d2-ce7df443e2da admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd25c5eab42c4d319217c324cb13acb8, user_id=c97a085ef91e4c8392091477e6596c57, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:53.781681 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 826/1626] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:47:53 2022] POST /identity/v3/users => generated 309 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:47:53.784586 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbea836-fbe1-4a99-a214-42943ffe0483 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:53.787510 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23859ea-8329-4672-a3d2-ce7df443e2da admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:53.820205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbea836-fbe1-4a99-a214-42943ffe0483 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:53.839108 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 801/1627] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:47:53 2022] PUT /identity/v3/projects/cd25c5eab42c4d319217c324cb13acb8/users/c97a085ef91e4c8392091477e6596c57/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:47:53.842100 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23859ea-8329-4672-a3d2-ce7df443e2da admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:54.051605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:54.059962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fda9cf-7529-4d8c-8d20-07ca3ec87981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:54.061650 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fda9cf-7529-4d8c-8d20-07ca3ec87981 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:54.061947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fda9cf-7529-4d8c-8d20-07ca3ec87981 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:54.062227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fda9cf-7529-4d8c-8d20-07ca3ec87981 admin admin] PATH_INFO: `/v3/projects/393a6c2119db45b59d82fba236dd420f/users/55193f07f44f4d5082c833f661db779f/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:54.094762 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23859ea-8329-4672-a3d2-ce7df443e2da admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:54.105723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fbf2d-cca4-42b4-aba6-a6602e67a4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:54.106980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fbf2d-cca4-42b4-aba6-a6602e67a4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:54.107665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fbf2d-cca4-42b4-aba6-a6602e67a4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:54.108121 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fbf2d-cca4-42b4-aba6-a6602e67a4a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:54.109874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fbf2d-cca4-42b4-aba6-a6602e67a4a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:54.115825 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fbf2d-cca4-42b4-aba6-a6602e67a4a2 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:54.121300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fda9cf-7529-4d8c-8d20-07ca3ec87981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=393a6c2119db45b59d82fba236dd420f, user_id=55193f07f44f4d5082c833f661db779f, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:54.127872 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fda9cf-7529-4d8c-8d20-07ca3ec87981 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:54.146766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 802/1628] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:54.149269 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fbf2d-cca4-42b4-aba6-a6602e67a4a2 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:54.164909 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 827/1629] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:47:53 2022] PUT /identity/v3/projects/393a6c2119db45b59d82fba236dd420f/users/55193f07f44f4d5082c833f661db779f/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:47:54.165782 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fda9cf-7529-4d8c-8d20-07ca3ec87981 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:54.174466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fbf2d-cca4-42b4-aba6-a6602e67a4a2 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:54.318665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dffba-e67d-41a3-beca-a319301ca5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:54.320457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dffba-e67d-41a3-beca-a319301ca5ca admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:54.320719 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dffba-e67d-41a3-beca-a319301ca5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:54.321228 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dffba-e67d-41a3-beca-a319301ca5ca admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:54.322367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dffba-e67d-41a3-beca-a319301ca5ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:54.324463 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dffba-e67d-41a3-beca-a319301ca5ca admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:54.343252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 803/1630] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:54.344424 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dffba-e67d-41a3-beca-a319301ca5ca admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:54.360324 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dffba-e67d-41a3-beca-a319301ca5ca admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:54.370203 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db414419-3007-4816-afea-f8bea8728446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:54.371883 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db414419-3007-4816-afea-f8bea8728446 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:54.372307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db414419-3007-4816-afea-f8bea8728446 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:54.373041 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db414419-3007-4816-afea-f8bea8728446 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:54.374965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db414419-3007-4816-afea-f8bea8728446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:54.393898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db414419-3007-4816-afea-f8bea8728446 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:54.408719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fda9cf-7529-4d8c-8d20-07ca3ec87981 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:54.412197 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 804/1631] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:54.415751 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a136a20-92e6-4241-a761-2b4a72790a67 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:54.416157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a136a20-92e6-4241-a761-2b4a72790a67 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:54.416419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a136a20-92e6-4241-a761-2b4a72790a67 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:54.420779 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 805/1632] 10.222.0.20 () {54 vars in 912 bytes} [Mon Jun 27 15:47:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:47:54.422555 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a136a20-92e6-4241-a761-2b4a72790a67 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:54.428468 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533b9ac-bace-4cdf-b000-f50aa1299de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:54.430001 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533b9ac-bace-4cdf-b000-f50aa1299de0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:54.430371 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533b9ac-bace-4cdf-b000-f50aa1299de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:54.430561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533b9ac-bace-4cdf-b000-f50aa1299de0 admin admin] PATH_INFO: `/v3/projects/d327e3a386cc4093a59b3390bb7b05a2/users/7661a47ab9a74a45b41ba8aaf61b9ee4/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:54.445420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a136a20-92e6-4241-a761-2b4a72790a67 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:54.451411 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccead96-b992-4abd-b8f2-7b8de586994c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:54.453891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccead96-b992-4abd-b8f2-7b8de586994c admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:54.454505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccead96-b992-4abd-b8f2-7b8de586994c admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:54.455228 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccead96-b992-4abd-b8f2-7b8de586994c admin admin] PATH_INFO: `/v3/projects/cd25c5eab42c4d319217c324cb13acb8/users/c97a085ef91e4c8392091477e6596c57/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:54.462596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccead96-b992-4abd-b8f2-7b8de586994c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd25c5eab42c4d319217c324cb13acb8, user_id=c97a085ef91e4c8392091477e6596c57, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:54.466371 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccead96-b992-4abd-b8f2-7b8de586994c admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:54.482214 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533b9ac-bace-4cdf-b000-f50aa1299de0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d327e3a386cc4093a59b3390bb7b05a2, user_id=7661a47ab9a74a45b41ba8aaf61b9ee4, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:54.494666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533b9ac-bace-4cdf-b000-f50aa1299de0 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:54.496401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 806/1633] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:47:54 2022] PUT /identity/v3/projects/cd25c5eab42c4d319217c324cb13acb8/users/c97a085ef91e4c8392091477e6596c57/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:47:54.497712 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccead96-b992-4abd-b8f2-7b8de586994c admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:54.513554 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:54.527326 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 828/1634] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:47:54 2022] PUT /identity/v3/projects/d327e3a386cc4093a59b3390bb7b05a2/users/7661a47ab9a74a45b41ba8aaf61b9ee4/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:47:54.530062 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d95e2-94bf-4500-8f86-9d34ffa20fab None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:54.530352 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d95e2-94bf-4500-8f86-9d34ffa20fab None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:54.530631 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d95e2-94bf-4500-8f86-9d34ffa20fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:54.606883 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-d76d95e2-94bf-4500-8f86-9d34ffa20fab None None] MFA Rules not processed for user `1c8f6262cf24419daf803ae71aed24f5`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:54.608150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-d76d95e2-94bf-4500-8f86-9d34ffa20fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:54.627284 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccead96-b992-4abd-b8f2-7b8de586994c admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:54.759837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa8b85-1dcc-4c6f-92c8-dfb705b99400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:54.763479 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa8b85-1dcc-4c6f-92c8-dfb705b99400 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:54.763817 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa8b85-1dcc-4c6f-92c8-dfb705b99400 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:54.764095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa8b85-1dcc-4c6f-92c8-dfb705b99400 admin admin] PATH_INFO: `/v3/projects/393a6c2119db45b59d82fba236dd420f/users/55193f07f44f4d5082c833f661db779f/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:54.768750 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa8b85-1dcc-4c6f-92c8-dfb705b99400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=393a6c2119db45b59d82fba236dd420f, user_id=55193f07f44f4d5082c833f661db779f, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:54.772768 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa8b85-1dcc-4c6f-92c8-dfb705b99400 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:54.787012 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 829/1635] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jun 27 15:47:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:54.798273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:54.798453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c090852-c573-45d8-939c-946297f15031 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:54.798622 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c090852-c573-45d8-939c-946297f15031 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:54.798966 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c090852-c573-45d8-939c-946297f15031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:54.824663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 807/1636] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:47:54 2022] PUT /identity/v3/projects/393a6c2119db45b59d82fba236dd420f/users/55193f07f44f4d5082c833f661db779f/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:47:54.826951 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa8b85-1dcc-4c6f-92c8-dfb705b99400 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:54.872427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-1c090852-c573-45d8-939c-946297f15031 None None] MFA Rules not processed for user `c97a085ef91e4c8392091477e6596c57`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:54.873712 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1c090852-c573-45d8-939c-946297f15031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:54.903215 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa8b85-1dcc-4c6f-92c8-dfb705b99400 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:54.908862 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fe840-180b-4e22-a1cc-d50bcffd8dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:54.909674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fe840-180b-4e22-a1cc-d50bcffd8dfb admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:54.910044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fe840-180b-4e22-a1cc-d50bcffd8dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:54.910392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fe840-180b-4e22-a1cc-d50bcffd8dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:54.911208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9fe840-180b-4e22-a1cc-d50bcffd8dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:54.912952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9fe840-180b-4e22-a1cc-d50bcffd8dfb admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:54.927532 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 808/1637] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:47:54.928290 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fe840-180b-4e22-a1cc-d50bcffd8dfb admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:54.998128 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fe840-180b-4e22-a1cc-d50bcffd8dfb admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:55.003982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff74ce1-a86a-46b1-8a11-0e45f550ca53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:55.004741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff74ce1-a86a-46b1-8a11-0e45f550ca53 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:55.005126 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff74ce1-a86a-46b1-8a11-0e45f550ca53 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:55.005467 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff74ce1-a86a-46b1-8a11-0e45f550ca53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:55.049362 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 830/1638] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:54 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:55.055141 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f238e3-5da0-4a8e-8e4f-940f3fb5ef42 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:55.055762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f238e3-5da0-4a8e-8e4f-940f3fb5ef42 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:55.056391 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f238e3-5da0-4a8e-8e4f-940f3fb5ef42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:55.107597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-61f238e3-5da0-4a8e-8e4f-940f3fb5ef42 None None] MFA Rules not processed for user `55193f07f44f4d5082c833f661db779f`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:55.109341 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-61f238e3-5da0-4a8e-8e4f-940f3fb5ef42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:55.118539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff74ce1-a86a-46b1-8a11-0e45f550ca53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:55.121780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff74ce1-a86a-46b1-8a11-0e45f550ca53 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:55.150655 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 809/1639] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:47:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:47:55.152864 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff74ce1-a86a-46b1-8a11-0e45f550ca53 service cinder] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:55.192807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff74ce1-a86a-46b1-8a11-0e45f550ca53 service cinder] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:55.205217 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a047b-25ed-4b3e-990c-b64709488713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:55.207510 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a047b-25ed-4b3e-990c-b64709488713 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:55.208926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a047b-25ed-4b3e-990c-b64709488713 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:55.209461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a047b-25ed-4b3e-990c-b64709488713 admin admin] PATH_INFO: `/v3/projects/d327e3a386cc4093a59b3390bb7b05a2/users/7661a47ab9a74a45b41ba8aaf61b9ee4/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:55.220294 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823a047b-25ed-4b3e-990c-b64709488713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d327e3a386cc4093a59b3390bb7b05a2, user_id=7661a47ab9a74a45b41ba8aaf61b9ee4, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:55.223873 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823a047b-25ed-4b3e-990c-b64709488713 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:55.251058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 810/1640] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:47:55 2022] PUT /identity/v3/projects/d327e3a386cc4093a59b3390bb7b05a2/users/7661a47ab9a74a45b41ba8aaf61b9ee4/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:47:55.253351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a047b-25ed-4b3e-990c-b64709488713 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:55.318780 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:55.329422 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 831/1641] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:55 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:55.331890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5903db-2a64-4b1a-a7d3-754b2a63d5b0 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:55.331890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5903db-2a64-4b1a-a7d3-754b2a63d5b0 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:55.331890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5903db-2a64-4b1a-a7d3-754b2a63d5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:55.348324 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:55.399714 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-9a5903db-2a64-4b1a-a7d3-754b2a63d5b0 None None] MFA Rules not processed for user `7661a47ab9a74a45b41ba8aaf61b9ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:55.403122 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-9a5903db-2a64-4b1a-a7d3-754b2a63d5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:55.422049 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a047b-25ed-4b3e-990c-b64709488713 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:55.431001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98ae23-f194-4fa1-a516-56025bff4af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:55.433493 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98ae23-f194-4fa1-a516-56025bff4af1 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:55.434741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98ae23-f194-4fa1-a516-56025bff4af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:55.435556 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98ae23-f194-4fa1-a516-56025bff4af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:55.661829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98ae23-f194-4fa1-a516-56025bff4af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:55.665548 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98ae23-f194-4fa1-a516-56025bff4af1 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:55.666206 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 832/1642] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:55 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:55.689277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 811/1643] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:47:55 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:47:55.760315 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5903db-2a64-4b1a-a7d3-754b2a63d5b0 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:55.777768 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5903db-2a64-4b1a-a7d3-754b2a63d5b0 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:55.786285 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3213f-2eef-4791-9b4b-a0aa60739701 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:55.789830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3213f-2eef-4791-9b4b-a0aa60739701 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:55.790308 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3213f-2eef-4791-9b4b-a0aa60739701 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:55.790763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3213f-2eef-4791-9b4b-a0aa60739701 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:55.801357 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98ae23-f194-4fa1-a516-56025bff4af1 service cinder] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:55.815657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98ae23-f194-4fa1-a516-56025bff4af1 service cinder] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:55.829085 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa772f-d29b-4e56-b5d5-2846512f5f65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:55.830801 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa772f-d29b-4e56-b5d5-2846512f5f65 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:55.832324 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa772f-d29b-4e56-b5d5-2846512f5f65 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:55.832749 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa772f-d29b-4e56-b5d5-2846512f5f65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:55.861665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa772f-d29b-4e56-b5d5-2846512f5f65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:55.863059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa772f-d29b-4e56-b5d5-2846512f5f65 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:55.896162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 812/1644] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:47:55 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:47:55.988648 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf3213f-2eef-4791-9b4b-a0aa60739701 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:55.999189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf3213f-2eef-4791-9b4b-a0aa60739701 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:56.043939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 833/1645] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:47:55 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:47:56.446133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa772f-d29b-4e56-b5d5-2846512f5f65 service cinder] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:56.461129 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa772f-d29b-4e56-b5d5-2846512f5f65 service cinder] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:56.474363 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822d352-39d0-4988-ae66-621f29cbddbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:56.475353 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822d352-39d0-4988-ae66-621f29cbddbb service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:56.475655 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822d352-39d0-4988-ae66-621f29cbddbb service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:56.475951 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822d352-39d0-4988-ae66-621f29cbddbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:56.620519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822d352-39d0-4988-ae66-621f29cbddbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:56.622055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822d352-39d0-4988-ae66-621f29cbddbb service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:56.662587 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 813/1646] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:47:56 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:47:56.701573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3213f-2eef-4791-9b4b-a0aa60739701 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:56.712282 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3213f-2eef-4791-9b4b-a0aa60739701 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:56.717433 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e433c2-6208-48ae-936f-d98bc0fe2949 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:56.718185 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e433c2-6208-48ae-936f-d98bc0fe2949 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:56.718473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e433c2-6208-48ae-936f-d98bc0fe2949 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:56.718758 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e433c2-6208-48ae-936f-d98bc0fe2949 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:56.814455 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86183499-ed78-4282-9740-1daf8e02f386 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:56.814750 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86183499-ed78-4282-9740-1daf8e02f386 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:56.815041 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86183499-ed78-4282-9740-1daf8e02f386 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:56.817325 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 814/1647] 10.222.0.20 () {54 vars in 961 bytes} [Mon Jun 27 15:47:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:47:56.827435 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e433c2-6208-48ae-936f-d98bc0fe2949 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:56.829054 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e433c2-6208-48ae-936f-d98bc0fe2949 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:56.858104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 834/1648] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:47:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:47:56.863719 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b426d02-ad01-4f3c-9c51-d798c1f29805 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:56.864285 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b426d02-ad01-4f3c-9c51-d798c1f29805 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:56.864736 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b426d02-ad01-4f3c-9c51-d798c1f29805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:56.905000 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-3b426d02-ad01-4f3c-9c51-d798c1f29805 None None] MFA Rules not processed for user `1c8f6262cf24419daf803ae71aed24f5`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:56.909205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3b426d02-ad01-4f3c-9c51-d798c1f29805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:57.057464 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:57.110495 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 815/1649] 10.222.0.20 () {58 vars in 1058 bytes} [Mon Jun 27 15:47:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:57.112840 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd32c1-9e37-4e5f-bfbe-0c310882787f None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:57.113188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd32c1-9e37-4e5f-bfbe-0c310882787f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:57.113479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd32c1-9e37-4e5f-bfbe-0c310882787f None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:57.116373 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 835/1650] 10.222.0.20 () {54 vars in 912 bytes} [Mon Jun 27 15:47:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:47:57.122606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dba2b9-31ba-4854-970a-ee5e88cb290d None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:57.123001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dba2b9-31ba-4854-970a-ee5e88cb290d None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:57.123161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dba2b9-31ba-4854-970a-ee5e88cb290d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:57.166167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-04dba2b9-31ba-4854-970a-ee5e88cb290d None None] MFA Rules not processed for user `1c8f6262cf24419daf803ae71aed24f5`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:57.169049 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-04dba2b9-31ba-4854-970a-ee5e88cb290d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:57.191201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 816/1651] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jun 27 15:47:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:57.207279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd32c1-9e37-4e5f-bfbe-0c310882787f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:57.267453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:57.385013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd32c1-9e37-4e5f-bfbe-0c310882787f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:57.390419 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611fcf2-badf-4924-8e0c-30fc168ae34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:57.391328 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611fcf2-badf-4924-8e0c-30fc168ae34a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:57.391756 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611fcf2-badf-4924-8e0c-30fc168ae34a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:57.392039 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611fcf2-badf-4924-8e0c-30fc168ae34a admin admin] PATH_INFO: `/v3/users/c03dd300681e4929879f8e08fb100347` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:57.395099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8611fcf2-badf-4924-8e0c-30fc168ae34a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c03dd300681e4929879f8e08fb100347)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:57.397144 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8611fcf2-badf-4924-8e0c-30fc168ae34a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:57.493974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8611fcf2-badf-4924-8e0c-30fc168ae34a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c03dd300681e4929879f8e08fb100347'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:47:57.537342 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8611fcf2-badf-4924-8e0c-30fc168ae34a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c03dd300681e4929879f8e08fb100347'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:47:57.541893 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8611fcf2-badf-4924-8e0c-30fc168ae34a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c03dd300681e4929879f8e08fb100347'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:47:57.557372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8611fcf2-badf-4924-8e0c-30fc168ae34a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c03dd300681e4929879f8e08fb100347'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:47:57.581630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8611fcf2-badf-4924-8e0c-30fc168ae34a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c03dd300681e4929879f8e08fb100347'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:47:57.600488 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 836/1652] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:47:57 2022] DELETE /identity/v3/users/c03dd300681e4929879f8e08fb100347 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:47:57.605478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca56157-148e-488b-97f7-361c6dc42eed None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:57.606018 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca56157-148e-488b-97f7-361c6dc42eed None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:57.606120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca56157-148e-488b-97f7-361c6dc42eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:57.622148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611fcf2-badf-4924-8e0c-30fc168ae34a admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:57.626925 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8611fcf2-badf-4924-8e0c-30fc168ae34a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:57.661784 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-2ca56157-148e-488b-97f7-361c6dc42eed None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:57.666578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-2ca56157-148e-488b-97f7-361c6dc42eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:57.718393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611fcf2-badf-4924-8e0c-30fc168ae34a admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:57.724747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ac0bb-2b99-40dd-a3f3-e183071beae2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:57.726770 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ac0bb-2b99-40dd-a3f3-e183071beae2 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:57.727818 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ac0bb-2b99-40dd-a3f3-e183071beae2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:57.728271 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ac0bb-2b99-40dd-a3f3-e183071beae2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:57.730661 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0d1ac0bb-2b99-40dd-a3f3-e183071beae2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:57.842923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:47:57.903617 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ac0bb-2b99-40dd-a3f3-e183071beae2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:57.905183 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ac0bb-2b99-40dd-a3f3-e183071beae2 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:57.932756 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 837/1653] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:47:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:47:57.938930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 817/1654] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:57.958194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ac0bb-2b99-40dd-a3f3-e183071beae2 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:57.959641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0d1ac0bb-2b99-40dd-a3f3-e183071beae2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:58.149896 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ac0bb-2b99-40dd-a3f3-e183071beae2 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:58.168354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224082e-4b4a-4f3d-a948-b8cd32f98a55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:58.169045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224082e-4b4a-4f3d-a948-b8cd32f98a55 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:58.169334 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224082e-4b4a-4f3d-a948-b8cd32f98a55 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:58.169615 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224082e-4b4a-4f3d-a948-b8cd32f98a55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:58.207614 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224082e-4b4a-4f3d-a948-b8cd32f98a55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:58.209081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224082e-4b4a-4f3d-a948-b8cd32f98a55 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:58.236242 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 838/1655] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:47:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:47:58.639595 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc35c95-04e4-41e0-8a47-25317b9a923f None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:58.640151 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc35c95-04e4-41e0-8a47-25317b9a923f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:58.640151 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc35c95-04e4-41e0-8a47-25317b9a923f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:58.763048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-6bc35c95-04e4-41e0-8a47-25317b9a923f None None] MFA Rules not processed for user `3520bd4387a140f88558d5f6fe55986e`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:58.780865 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-6bc35c95-04e4-41e0-8a47-25317b9a923f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:58.967924 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 818/1656] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:47:58.980920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889e85c-ab71-4f50-9683-1ffb95d56075 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:58.982102 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889e85c-ab71-4f50-9683-1ffb95d56075 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:58.982567 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889e85c-ab71-4f50-9683-1ffb95d56075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:59.030843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-7889e85c-ab71-4f50-9683-1ffb95d56075 None None] MFA Rules not processed for user `3520bd4387a140f88558d5f6fe55986e`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:47:59.036873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-7889e85c-ab71-4f50-9683-1ffb95d56075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:59.060343 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 839/1657] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:47:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:47:59.073989 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc35c95-04e4-41e0-8a47-25317b9a923f None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:59.076857 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-6bc35c95-04e4-41e0-8a47-25317b9a923f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:59.118272 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc35c95-04e4-41e0-8a47-25317b9a923f None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:59.131224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee76290-6586-42dd-a438-9f8a22b87ffb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:59.131905 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee76290-6586-42dd-a438-9f8a22b87ffb service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:59.132249 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee76290-6586-42dd-a438-9f8a22b87ffb service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:59.132530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee76290-6586-42dd-a438-9f8a22b87ffb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:59.166225 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee76290-6586-42dd-a438-9f8a22b87ffb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:59.168051 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee76290-6586-42dd-a438-9f8a22b87ffb service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:59.193006 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 819/1658] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:47:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:47:59.380746 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889e85c-ab71-4f50-9683-1ffb95d56075 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:59.381866 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-7889e85c-ab71-4f50-9683-1ffb95d56075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:47:59.443567 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889e85c-ab71-4f50-9683-1ffb95d56075 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:59.459334 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f331430-eaf9-4b64-ba29-35a5c197d63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:59.463066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f331430-eaf9-4b64-ba29-35a5c197d63d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:59.463331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f331430-eaf9-4b64-ba29-35a5c197d63d admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:59.463619 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f331430-eaf9-4b64-ba29-35a5c197d63d admin admin] PATH_INFO: `/v3/projects/bfe198915ace4d5c87125cbfbb9f7cba` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:59.470145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f331430-eaf9-4b64-ba29-35a5c197d63d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfe198915ace4d5c87125cbfbb9f7cba)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:59.471171 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: WARNING py.warnings [None req-9f331430-eaf9-4b64-ba29-35a5c197d63d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:47:59.471171 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: warnings.warn(msg) Jun 27 15:47:59.471171 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: Jun 27 15:47:59.473822 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f331430-eaf9-4b64-ba29-35a5c197d63d admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:59.651144 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9f331430-eaf9-4b64-ba29-35a5c197d63d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfe198915ace4d5c87125cbfbb9f7cba', 'request_id': 'req-9f331430-eaf9-4b64-ba29-35a5c197d63d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:47:59.702999 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9f331430-eaf9-4b64-ba29-35a5c197d63d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfe198915ace4d5c87125cbfbb9f7cba', 'request_id': 'req-9f331430-eaf9-4b64-ba29-35a5c197d63d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:47:59.714229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9f331430-eaf9-4b64-ba29-35a5c197d63d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfe198915ace4d5c87125cbfbb9f7cba', 'request_id': 'req-9f331430-eaf9-4b64-ba29-35a5c197d63d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:47:59.735791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9f331430-eaf9-4b64-ba29-35a5c197d63d admin admin] The token cache is being invalidate because project bfe198915ace4d5c87125cbfbb9f7cba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:47:59.744137 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9f331430-eaf9-4b64-ba29-35a5c197d63d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:47:59.745203 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9f331430-eaf9-4b64-ba29-35a5c197d63d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:47:59.747403 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 840/1659] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:47:59 2022] DELETE /identity/v3/projects/bfe198915ace4d5c87125cbfbb9f7cba => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:47:59.751347 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee76290-6586-42dd-a438-9f8a22b87ffb service nova] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:47:59.762881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee76290-6586-42dd-a438-9f8a22b87ffb service nova] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:47:59.889599 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513efbf-84fa-4360-abff-c084306228e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:47:59.893837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513efbf-84fa-4360-abff-c084306228e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:47:59.894530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513efbf-84fa-4360-abff-c084306228e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:47:59.897443 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513efbf-84fa-4360-abff-c084306228e7 admin admin] PATH_INFO: `/v3/users/7661a47ab9a74a45b41ba8aaf61b9ee4` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:47:59.901572 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513efbf-84fa-4360-abff-c084306228e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7661a47ab9a74a45b41ba8aaf61b9ee4)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:47:59.903343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: WARNING py.warnings [None req-d513efbf-84fa-4360-abff-c084306228e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 15:47:59.903343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: warnings.warn(msg) Jun 27 15:47:59.903343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: Jun 27 15:47:59.905381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513efbf-84fa-4360-abff-c084306228e7 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:47:59.907841 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:00.072059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d2f62-0166-41c1-9dd8-ef17dcc24048 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:00.072557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d2f62-0166-41c1-9dd8-ef17dcc24048 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:00.072964 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d2f62-0166-41c1-9dd8-ef17dcc24048 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:00.074552 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 841/1660] 10.222.0.20 () {54 vars in 912 bytes} [Mon Jun 27 15:48:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:48:00.081319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea39e2-8afc-40c6-a177-4f6029a73543 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:00.081753 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea39e2-8afc-40c6-a177-4f6029a73543 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:00.082206 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea39e2-8afc-40c6-a177-4f6029a73543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:00.114251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d513efbf-84fa-4360-abff-c084306228e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7661a47ab9a74a45b41ba8aaf61b9ee4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:00.150115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-05ea39e2-8afc-40c6-a177-4f6029a73543 None None] MFA Rules not processed for user `42dee639e2834d99b7468f911c37cf8f`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:00.151559 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-05ea39e2-8afc-40c6-a177-4f6029a73543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:00.180834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d513efbf-84fa-4360-abff-c084306228e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7661a47ab9a74a45b41ba8aaf61b9ee4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:00.198536 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d513efbf-84fa-4360-abff-c084306228e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7661a47ab9a74a45b41ba8aaf61b9ee4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:00.199214 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d513efbf-84fa-4360-abff-c084306228e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7661a47ab9a74a45b41ba8aaf61b9ee4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:00.202872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d513efbf-84fa-4360-abff-c084306228e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7661a47ab9a74a45b41ba8aaf61b9ee4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:00.246697 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:00.248421 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 820/1661] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:47:59 2022] DELETE /identity/v3/users/7661a47ab9a74a45b41ba8aaf61b9ee4 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:00.272595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 842/1662] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jun 27 15:48:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:48:00.303202 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513efbf-84fa-4360-abff-c084306228e7 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:00.307225 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d513efbf-84fa-4360-abff-c084306228e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:00.524614 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513efbf-84fa-4360-abff-c084306228e7 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:00.539628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a958d7-073f-44ce-8556-5b493f6429e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:00.540545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a958d7-073f-44ce-8556-5b493f6429e5 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:00.540931 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a958d7-073f-44ce-8556-5b493f6429e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:00.541314 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a958d7-073f-44ce-8556-5b493f6429e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:00.559585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a958d7-073f-44ce-8556-5b493f6429e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:00.561466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a958d7-073f-44ce-8556-5b493f6429e5 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:00.583575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 821/1663] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:01.468737 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea39e2-8afc-40c6-a177-4f6029a73543 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:01.470292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-05ea39e2-8afc-40c6-a177-4f6029a73543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:01.512514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea39e2-8afc-40c6-a177-4f6029a73543 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:01.718721 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31352473-9f95-4811-aca8-5d05e4efd163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:01.719799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31352473-9f95-4811-aca8-5d05e4efd163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:01.723952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31352473-9f95-4811-aca8-5d05e4efd163 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:01.725029 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31352473-9f95-4811-aca8-5d05e4efd163 admin admin] PATH_INFO: `/v3/projects/d327e3a386cc4093a59b3390bb7b05a2` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:01.728182 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31352473-9f95-4811-aca8-5d05e4efd163 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d327e3a386cc4093a59b3390bb7b05a2)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:01.735921 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31352473-9f95-4811-aca8-5d05e4efd163 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:01.871991 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-31352473-9f95-4811-aca8-5d05e4efd163 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd327e3a386cc4093a59b3390bb7b05a2', 'request_id': 'req-31352473-9f95-4811-aca8-5d05e4efd163'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:01.887501 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-31352473-9f95-4811-aca8-5d05e4efd163 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd327e3a386cc4093a59b3390bb7b05a2', 'request_id': 'req-31352473-9f95-4811-aca8-5d05e4efd163'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:01.909411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-31352473-9f95-4811-aca8-5d05e4efd163 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd327e3a386cc4093a59b3390bb7b05a2', 'request_id': 'req-31352473-9f95-4811-aca8-5d05e4efd163'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:01.935505 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-31352473-9f95-4811-aca8-5d05e4efd163 admin admin] The token cache is being invalidate because project d327e3a386cc4093a59b3390bb7b05a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:01.935906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-31352473-9f95-4811-aca8-5d05e4efd163 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:01.936726 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-31352473-9f95-4811-aca8-5d05e4efd163 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:01.938988 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 843/1664] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:01 2022] DELETE /identity/v3/projects/d327e3a386cc4093a59b3390bb7b05a2 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:01.945575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3493b-63fd-48c3-ac27-a7a749eb8101 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:01.946618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3493b-63fd-48c3-ac27-a7a749eb8101 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:01.947146 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3493b-63fd-48c3-ac27-a7a749eb8101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:01.981291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-63e3493b-63fd-48c3-ac27-a7a749eb8101 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:01.982772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-63e3493b-63fd-48c3-ac27-a7a749eb8101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:02.094374 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dac408-1183-4900-9631-ca256e48c4c7 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:02.094679 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dac408-1183-4900-9631-ca256e48c4c7 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:02.094918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dac408-1183-4900-9631-ca256e48c4c7 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:02.097270 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 844/1665] 10.222.0.20 () {54 vars in 960 bytes} [Mon Jun 27 15:48:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:48:02.110003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ae8c3-837e-4656-a5be-44ada725d369 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:02.110282 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ae8c3-837e-4656-a5be-44ada725d369 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:02.110561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ae8c3-837e-4656-a5be-44ada725d369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:02.180817 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-179ae8c3-837e-4656-a5be-44ada725d369 None None] MFA Rules not processed for user `ffafb68f1bb941588a5217ad83ad907c`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:02.187395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-179ae8c3-837e-4656-a5be-44ada725d369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:02.207724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 822/1666] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:02.209656 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dd924-a182-4d48-9433-78c55ded296f None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:02.212787 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dd924-a182-4d48-9433-78c55ded296f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:02.213834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dd924-a182-4d48-9433-78c55ded296f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:02.266895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-931dd924-a182-4d48-9433-78c55ded296f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:02.268566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-931dd924-a182-4d48-9433-78c55ded296f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:02.293413 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 823/1667] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:02.301660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931dd924-a182-4d48-9433-78c55ded296f None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:02.311622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931dd924-a182-4d48-9433-78c55ded296f None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:02.319327 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdd34f-4fa2-4d3e-9b59-1d3925e51a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:02.320908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdd34f-4fa2-4d3e-9b59-1d3925e51a6d admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:02.321339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdd34f-4fa2-4d3e-9b59-1d3925e51a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:02.321750 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdd34f-4fa2-4d3e-9b59-1d3925e51a6d admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:02.322806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cdd34f-4fa2-4d3e-9b59-1d3925e51a6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:48:02.323381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cdd34f-4fa2-4d3e-9b59-1d3925e51a6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:02.325934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cdd34f-4fa2-4d3e-9b59-1d3925e51a6d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:02.345455 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 824/1668] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:48:02.346480 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdd34f-4fa2-4d3e-9b59-1d3925e51a6d admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:02.353388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:02.360352 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdd34f-4fa2-4d3e-9b59-1d3925e51a6d admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:02.370092 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 845/1669] 10.222.0.20 () {58 vars in 1057 bytes} [Mon Jun 27 15:48:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 27 15:48:02.374284 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432cc3a1-ae4b-4ee1-91f0-68e82f8e46b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:02.375046 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cc3a1-ae4b-4ee1-91f0-68e82f8e46b8 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:02.375406 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cc3a1-ae4b-4ee1-91f0-68e82f8e46b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:02.375852 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cc3a1-ae4b-4ee1-91f0-68e82f8e46b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:02.376632 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432cc3a1-ae4b-4ee1-91f0-68e82f8e46b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:02.378885 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ae8c3-837e-4656-a5be-44ada725d369 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:02.383737 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432cc3a1-ae4b-4ee1-91f0-68e82f8e46b8 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:02.405756 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ae8c3-837e-4656-a5be-44ada725d369 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:02.420555 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04ba02-cf66-4208-806a-bf516f2f3888 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffafb68f1bb941588a5217ad83ad907c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:02.423134 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04ba02-cf66-4208-806a-bf516f2f3888 service placement] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:02.423730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04ba02-cf66-4208-806a-bf516f2f3888 service placement] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:02.424613 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04ba02-cf66-4208-806a-bf516f2f3888 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:02.427381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1b04ba02-cf66-4208-806a-bf516f2f3888 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:02.453772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 825/1670] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:02 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:02.456525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432cc3a1-ae4b-4ee1-91f0-68e82f8e46b8 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:02.470436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04ba02-cf66-4208-806a-bf516f2f3888 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:02.477932 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04ba02-cf66-4208-806a-bf516f2f3888 service placement] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:02.504237 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 846/1671] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:02.621884 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432cc3a1-ae4b-4ee1-91f0-68e82f8e46b8 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:02.628876 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5e03c-1cec-401e-b6ea-898c156f1893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:02.629827 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5e03c-1cec-401e-b6ea-898c156f1893 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:02.630209 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5e03c-1cec-401e-b6ea-898c156f1893 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:02.630816 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5e03c-1cec-401e-b6ea-898c156f1893 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:02.631823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da5e03c-1cec-401e-b6ea-898c156f1893 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:02.635466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da5e03c-1cec-401e-b6ea-898c156f1893 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:02.674324 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 826/1672] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:02 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:02.679540 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04ba02-cf66-4208-806a-bf516f2f3888 service placement] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:02.699509 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04ba02-cf66-4208-806a-bf516f2f3888 service placement] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:02.716334 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf436c2-035e-4b06-9f5d-b068085a4b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:02.717030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf436c2-035e-4b06-9f5d-b068085a4b01 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:02.717449 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf436c2-035e-4b06-9f5d-b068085a4b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:02.717606 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf436c2-035e-4b06-9f5d-b068085a4b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:02.718371 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf436c2-035e-4b06-9f5d-b068085a4b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:02.723268 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf436c2-035e-4b06-9f5d-b068085a4b01 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:02.726340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:02.787096 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 847/1673] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:02.791758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5e03c-1cec-401e-b6ea-898c156f1893 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:02.808470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5e03c-1cec-401e-b6ea-898c156f1893 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:02.826882 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39e700-5112-4c0d-9452-3f8c0a143563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:02.827811 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39e700-5112-4c0d-9452-3f8c0a143563 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:02.828185 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39e700-5112-4c0d-9452-3f8c0a143563 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:02.828558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39e700-5112-4c0d-9452-3f8c0a143563 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:02.829425 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39e700-5112-4c0d-9452-3f8c0a143563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:02.831310 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39e700-5112-4c0d-9452-3f8c0a143563 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:02.862304 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 827/1674] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:02.867324 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf436c2-035e-4b06-9f5d-b068085a4b01 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:02.881821 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf436c2-035e-4b06-9f5d-b068085a4b01 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:02.894287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4a8fc-dc00-4a31-b5d0-5a9ec8cb48c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:02.895397 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4a8fc-dc00-4a31-b5d0-5a9ec8cb48c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:02.895830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4a8fc-dc00-4a31-b5d0-5a9ec8cb48c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:02.896240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4a8fc-dc00-4a31-b5d0-5a9ec8cb48c7 admin admin] PATH_INFO: `/v3/projects/0f0f10f56afc411abcf332788db6ff2a/users/36f4ef810e2c47dea45de86774b23f29/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:02.917669 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4a8fc-dc00-4a31-b5d0-5a9ec8cb48c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0f10f56afc411abcf332788db6ff2a, user_id=36f4ef810e2c47dea45de86774b23f29, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:02.921072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4a8fc-dc00-4a31-b5d0-5a9ec8cb48c7 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:02.955924 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 848/1675] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:02 2022] PUT /identity/v3/projects/0f0f10f56afc411abcf332788db6ff2a/users/36f4ef810e2c47dea45de86774b23f29/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:02.966849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd3380-d930-47da-b132-1e7a1398e7af None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:02.967420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd3380-d930-47da-b132-1e7a1398e7af None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:02.967861 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd3380-d930-47da-b132-1e7a1398e7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:03.079005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-72bd3380-d930-47da-b132-1e7a1398e7af None None] MFA Rules not processed for user `36f4ef810e2c47dea45de86774b23f29`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:03.080248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-72bd3380-d930-47da-b132-1e7a1398e7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:03.247522 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 828/1676] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:03.249297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4a8fc-dc00-4a31-b5d0-5a9ec8cb48c7 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:03.469162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4a8fc-dc00-4a31-b5d0-5a9ec8cb48c7 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:03.488397 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39e577-89c7-4b62-8970-b56597b527d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:03.489455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39e577-89c7-4b62-8970-b56597b527d3 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:03.491097 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39e577-89c7-4b62-8970-b56597b527d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:03.496389 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39e577-89c7-4b62-8970-b56597b527d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:03.510851 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c39e577-89c7-4b62-8970-b56597b527d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:03.513852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c39e577-89c7-4b62-8970-b56597b527d3 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:03.574380 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 849/1677] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:03 2022] POST /identity/v3/projects => generated 373 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:03.578097 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd3380-d930-47da-b132-1e7a1398e7af None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:03.598009 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd3380-d930-47da-b132-1e7a1398e7af None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:03.792247 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:03.845393 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9a900-d9e9-4f00-bec8-b4cecf24e196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:03.850040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9a900-d9e9-4f00-bec8-b4cecf24e196 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:03.852320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9a900-d9e9-4f00-bec8-b4cecf24e196 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:03.855793 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9a900-d9e9-4f00-bec8-b4cecf24e196 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:03.857011 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9a900-d9e9-4f00-bec8-b4cecf24e196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:03.872243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9a900-d9e9-4f00-bec8-b4cecf24e196 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:03.955881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 829/1678] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:03 2022] POST /identity/v3/users => generated 293 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:03.957396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39e577-89c7-4b62-8970-b56597b527d3 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:03.979776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39e577-89c7-4b62-8970-b56597b527d3 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:04.000972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f668e-129a-4393-b921-f38a3389f0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:04.005910 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f668e-129a-4393-b921-f38a3389f0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:04.006220 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f668e-129a-4393-b921-f38a3389f0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:04.012026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f668e-129a-4393-b921-f38a3389f0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:04.012890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f668e-129a-4393-b921-f38a3389f0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:04.023481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f668e-129a-4393-b921-f38a3389f0e7 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:04.068364 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9a900-d9e9-4f00-bec8-b4cecf24e196 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:04.069726 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 850/1679] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:48:04.088536 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9a900-d9e9-4f00-bec8-b4cecf24e196 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:04.099849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4c4e1-0fd9-43fe-9a67-4746028744cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:04.101480 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4c4e1-0fd9-43fe-9a67-4746028744cd admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:04.101973 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4c4e1-0fd9-43fe-9a67-4746028744cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:04.102410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4c4e1-0fd9-43fe-9a67-4746028744cd admin admin] PATH_INFO: `/v3/projects/ce922fd498c84029a0123e1866b31dc5/users/6e29b9588d4a4ac092acda9dcbaa67bf/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:04.138841 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4c4e1-0fd9-43fe-9a67-4746028744cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce922fd498c84029a0123e1866b31dc5, user_id=6e29b9588d4a4ac092acda9dcbaa67bf, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:04.146692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4c4e1-0fd9-43fe-9a67-4746028744cd admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:04.189150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 830/1680] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:04 2022] PUT /identity/v3/projects/ce922fd498c84029a0123e1866b31dc5/users/6e29b9588d4a4ac092acda9dcbaa67bf/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:04.193285 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f668e-129a-4393-b921-f38a3389f0e7 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:04.212797 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f668e-129a-4393-b921-f38a3389f0e7 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:04.372869 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7b83b-9550-48ba-a5bf-7779d8f3f1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:04.373563 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7b83b-9550-48ba-a5bf-7779d8f3f1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:04.373879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7b83b-9550-48ba-a5bf-7779d8f3f1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:04.374161 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7b83b-9550-48ba-a5bf-7779d8f3f1b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:04.374906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7b83b-9550-48ba-a5bf-7779d8f3f1b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:04.376986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7b83b-9550-48ba-a5bf-7779d8f3f1b3 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:04.409410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4c4e1-0fd9-43fe-9a67-4746028744cd admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:04.416406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 851/1681] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:04.424858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4c4e1-0fd9-43fe-9a67-4746028744cd admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:04.438758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de73a7a-faee-4d4f-950e-9ffaed70048d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:04.439683 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de73a7a-faee-4d4f-950e-9ffaed70048d admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:04.440041 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de73a7a-faee-4d4f-950e-9ffaed70048d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:04.440329 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de73a7a-faee-4d4f-950e-9ffaed70048d admin admin] PATH_INFO: `/v3/projects/ce922fd498c84029a0123e1866b31dc5/users/6e29b9588d4a4ac092acda9dcbaa67bf/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:04.442854 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de73a7a-faee-4d4f-950e-9ffaed70048d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce922fd498c84029a0123e1866b31dc5, user_id=6e29b9588d4a4ac092acda9dcbaa67bf, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:04.445837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de73a7a-faee-4d4f-950e-9ffaed70048d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:04.468379 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 831/1682] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:04 2022] PUT /identity/v3/projects/ce922fd498c84029a0123e1866b31dc5/users/6e29b9588d4a4ac092acda9dcbaa67bf/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:04.470809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b45f80-0e14-45fa-86dd-36d39f583fa8 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:04.471103 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b45f80-0e14-45fa-86dd-36d39f583fa8 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:04.471384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b45f80-0e14-45fa-86dd-36d39f583fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:04.528466 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-59b45f80-0e14-45fa-86dd-36d39f583fa8 None None] MFA Rules not processed for user `6e29b9588d4a4ac092acda9dcbaa67bf`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:04.532950 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-59b45f80-0e14-45fa-86dd-36d39f583fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:04.605205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:04.716085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 852/1683] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:04.724179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de73a7a-faee-4d4f-950e-9ffaed70048d admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:04.727473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-0de73a7a-faee-4d4f-950e-9ffaed70048d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:04.868894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de73a7a-faee-4d4f-950e-9ffaed70048d admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:04.872847 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52107f2-5b8d-4eee-aaac-6a61de789e75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:04.874478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52107f2-5b8d-4eee-aaac-6a61de789e75 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:04.874478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52107f2-5b8d-4eee-aaac-6a61de789e75 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:04.874478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52107f2-5b8d-4eee-aaac-6a61de789e75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:04.903923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52107f2-5b8d-4eee-aaac-6a61de789e75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:04.905485 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52107f2-5b8d-4eee-aaac-6a61de789e75 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:04.930990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 832/1684] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:48:05.713911 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe1098-184b-4a88-aa02-fadadaab9263 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:05.715194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe1098-184b-4a88-aa02-fadadaab9263 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:05.715665 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe1098-184b-4a88-aa02-fadadaab9263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:05.784626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-7dbe1098-184b-4a88-aa02-fadadaab9263 None None] MFA Rules not processed for user `d35c652539f443d4895b5f81461833ba`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:05.786574 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-7dbe1098-184b-4a88-aa02-fadadaab9263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:05.974328 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 853/1685] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:05.993849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52107f2-5b8d-4eee-aaac-6a61de789e75 service cinder] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:05.993849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d52107f2-5b8d-4eee-aaac-6a61de789e75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:06.107011 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe1098-184b-4a88-aa02-fadadaab9263 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:06.336076 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52107f2-5b8d-4eee-aaac-6a61de789e75 service cinder] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:06.358425 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c5a6b-0da7-44b0-846d-46921fde33e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:06.360547 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c5a6b-0da7-44b0-846d-46921fde33e3 service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:06.361796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c5a6b-0da7-44b0-846d-46921fde33e3 service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:06.371021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c5a6b-0da7-44b0-846d-46921fde33e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:06.371113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe1098-184b-4a88-aa02-fadadaab9263 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:06.386888 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603b4fc-2ea5-4948-b7e9-8c80049f4504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:06.388153 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603b4fc-2ea5-4948-b7e9-8c80049f4504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:06.388619 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603b4fc-2ea5-4948-b7e9-8c80049f4504 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:06.388970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603b4fc-2ea5-4948-b7e9-8c80049f4504 admin admin] PATH_INFO: `/v3/users/36f4ef810e2c47dea45de86774b23f29` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:06.390647 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603b4fc-2ea5-4948-b7e9-8c80049f4504 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36f4ef810e2c47dea45de86774b23f29)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:06.392530 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603b4fc-2ea5-4948-b7e9-8c80049f4504 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:06.396192 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c5a6b-0da7-44b0-846d-46921fde33e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:06.404278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c5a6b-0da7-44b0-846d-46921fde33e3 service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:06.473494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 833/1686] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:06.537672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8603b4fc-2ea5-4948-b7e9-8c80049f4504 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36f4ef810e2c47dea45de86774b23f29'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:06.566087 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:06.600290 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8603b4fc-2ea5-4948-b7e9-8c80049f4504 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36f4ef810e2c47dea45de86774b23f29'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:06.606138 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8603b4fc-2ea5-4948-b7e9-8c80049f4504 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36f4ef810e2c47dea45de86774b23f29'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:06.623555 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8603b4fc-2ea5-4948-b7e9-8c80049f4504 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36f4ef810e2c47dea45de86774b23f29'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:06.649106 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8603b4fc-2ea5-4948-b7e9-8c80049f4504 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36f4ef810e2c47dea45de86774b23f29'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:06.665425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 854/1687] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:06 2022] DELETE /identity/v3/users/36f4ef810e2c47dea45de86774b23f29 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:48:06.670164 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034ba1b-a01e-4bb2-8c21-3de8e78867bc None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:06.670530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034ba1b-a01e-4bb2-8c21-3de8e78867bc None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:06.670906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034ba1b-a01e-4bb2-8c21-3de8e78867bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:06.727258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-9034ba1b-a01e-4bb2-8c21-3de8e78867bc None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:06.728484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-9034ba1b-a01e-4bb2-8c21-3de8e78867bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:06.945516 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:07.130242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 834/1688] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:48:07.176362 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603b4fc-2ea5-4948-b7e9-8c80049f4504 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:07.178753 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8603b4fc-2ea5-4948-b7e9-8c80049f4504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:07.472798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603b4fc-2ea5-4948-b7e9-8c80049f4504 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:07.486031 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9262d80-0a7b-415f-8063-25dcb8c93f01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:07.487113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9262d80-0a7b-415f-8063-25dcb8c93f01 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:07.487546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9262d80-0a7b-415f-8063-25dcb8c93f01 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:07.487977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9262d80-0a7b-415f-8063-25dcb8c93f01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:07.520895 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9262d80-0a7b-415f-8063-25dcb8c93f01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:07.522617 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9262d80-0a7b-415f-8063-25dcb8c93f01 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:07.552412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 855/1689] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:07.582168 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034ba1b-a01e-4bb2-8c21-3de8e78867bc None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:07.582168 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-9034ba1b-a01e-4bb2-8c21-3de8e78867bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:07.608352 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034ba1b-a01e-4bb2-8c21-3de8e78867bc None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:07.631793 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e816f-3f5d-4dbf-b427-a6d307b100c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:07.637941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e816f-3f5d-4dbf-b427-a6d307b100c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:07.645021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e816f-3f5d-4dbf-b427-a6d307b100c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:07.645459 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e816f-3f5d-4dbf-b427-a6d307b100c0 admin admin] PATH_INFO: `/v3/users/61c7ee6276174c5f88e86810f632e690` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:07.662991 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e816f-3f5d-4dbf-b427-a6d307b100c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61c7ee6276174c5f88e86810f632e690)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:07.672683 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e816f-3f5d-4dbf-b427-a6d307b100c0 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:07.867540 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-586e816f-3f5d-4dbf-b427-a6d307b100c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61c7ee6276174c5f88e86810f632e690'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:07.909257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-586e816f-3f5d-4dbf-b427-a6d307b100c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61c7ee6276174c5f88e86810f632e690'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:07.927885 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-586e816f-3f5d-4dbf-b427-a6d307b100c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61c7ee6276174c5f88e86810f632e690'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:07.930239 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-586e816f-3f5d-4dbf-b427-a6d307b100c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61c7ee6276174c5f88e86810f632e690'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:07.931998 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-586e816f-3f5d-4dbf-b427-a6d307b100c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61c7ee6276174c5f88e86810f632e690'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:07.970002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 835/1690] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:07 2022] DELETE /identity/v3/users/61c7ee6276174c5f88e86810f632e690 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:07.979152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5866d-51b5-41c0-b01e-08b396fc82aa None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:07.979779 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5866d-51b5-41c0-b01e-08b396fc82aa None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:07.980155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5866d-51b5-41c0-b01e-08b396fc82aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:07.986559 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a4dc3-0d7d-4754-a424-c67e761be70e None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:07.986860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a4dc3-0d7d-4754-a424-c67e761be70e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:07.987144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a4dc3-0d7d-4754-a424-c67e761be70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:08.018605 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-41e5866d-51b5-41c0-b01e-08b396fc82aa None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:08.027653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-41e5866d-51b5-41c0-b01e-08b396fc82aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:08.071703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-e36a4dc3-0d7d-4754-a424-c67e761be70e None None] MFA Rules not processed for user `1a7a81da8acf40849a9ea86e2542ae6a`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:08.074253 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e36a4dc3-0d7d-4754-a424-c67e761be70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:08.230161 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 856/1691] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:08.250858 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90681061-3be5-43ca-9586-f2aa17a21722 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:08.251426 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90681061-3be5-43ca-9586-f2aa17a21722 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:08.251871 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90681061-3be5-43ca-9586-f2aa17a21722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:08.265956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 836/1692] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:08.267687 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a4dc3-0d7d-4754-a424-c67e761be70e None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:08.277322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e36a4dc3-0d7d-4754-a424-c67e761be70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:08.317037 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-90681061-3be5-43ca-9586-f2aa17a21722 None None] MFA Rules not processed for user `6655366e7eb84a26aa3f7fbe88af62b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:08.323461 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-90681061-3be5-43ca-9586-f2aa17a21722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:08.375068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:08.427074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a4dc3-0d7d-4754-a424-c67e761be70e None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:08.444192 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4991c-683f-4f23-8f03-6a3dcbbd071a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:08.448506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4991c-683f-4f23-8f03-6a3dcbbd071a service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:08.449433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4991c-683f-4f23-8f03-6a3dcbbd071a service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:08.450691 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4991c-683f-4f23-8f03-6a3dcbbd071a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:08.480128 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4991c-683f-4f23-8f03-6a3dcbbd071a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:08.485176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4991c-683f-4f23-8f03-6a3dcbbd071a service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:08.555306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 837/1693] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:08.563476 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d2e97-5dfa-40a4-b81b-8729235d93ec None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:08.563913 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d2e97-5dfa-40a4-b81b-8729235d93ec None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:08.565338 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d2e97-5dfa-40a4-b81b-8729235d93ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:08.611131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 857/1694] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jun 27 15:48:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:08.635545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-d85d2e97-5dfa-40a4-b81b-8729235d93ec None None] MFA Rules not processed for user `1a7a81da8acf40849a9ea86e2542ae6a`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:08.636851 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d85d2e97-5dfa-40a4-b81b-8729235d93ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:08.658873 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 838/1695] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:08.671975 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90681061-3be5-43ca-9586-f2aa17a21722 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:08.673547 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-90681061-3be5-43ca-9586-f2aa17a21722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:08.793307 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:08.901250 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d2e97-5dfa-40a4-b81b-8729235d93ec None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:08.906815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d85d2e97-5dfa-40a4-b81b-8729235d93ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:08.971185 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90681061-3be5-43ca-9586-f2aa17a21722 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:08.989130 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5bef07-4f73-4942-a69f-270faaace63f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:08.991940 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bef07-4f73-4942-a69f-270faaace63f service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:08.992922 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bef07-4f73-4942-a69f-270faaace63f service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:08.993859 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bef07-4f73-4942-a69f-270faaace63f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:09.019621 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5bef07-4f73-4942-a69f-270faaace63f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:09.022500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5bef07-4f73-4942-a69f-270faaace63f service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:09.055861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 858/1696] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:09.082135 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d2e97-5dfa-40a4-b81b-8729235d93ec None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:09.093883 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281636b-e85b-4763-8b45-da81ef54188c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e83ea69ded040448fe75caa221ad377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:09.094855 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281636b-e85b-4763-8b45-da81ef54188c service swift] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:09.095403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281636b-e85b-4763-8b45-da81ef54188c service swift] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:09.095840 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281636b-e85b-4763-8b45-da81ef54188c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:09.107790 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281636b-e85b-4763-8b45-da81ef54188c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:09.110506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281636b-e85b-4763-8b45-da81ef54188c service swift] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:09.148879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 839/1697] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jun 27 15:48:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 15:48:09.198907 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5bef07-4f73-4942-a69f-270faaace63f service nova] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:09.200467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-de5bef07-4f73-4942-a69f-270faaace63f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:09.222620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5bef07-4f73-4942-a69f-270faaace63f service nova] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:09.231743 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5822f92-34a8-4e2b-b303-d30b4bccd6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:09.235692 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5822f92-34a8-4e2b-b303-d30b4bccd6a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:09.236297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5822f92-34a8-4e2b-b303-d30b4bccd6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:09.240286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5822f92-34a8-4e2b-b303-d30b4bccd6a9 admin admin] PATH_INFO: `/v3/projects/0f0f10f56afc411abcf332788db6ff2a` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:09.245218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5822f92-34a8-4e2b-b303-d30b4bccd6a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f0f10f56afc411abcf332788db6ff2a)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:09.257888 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5822f92-34a8-4e2b-b303-d30b4bccd6a9 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:09.567847 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-f5822f92-34a8-4e2b-b303-d30b4bccd6a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f0f10f56afc411abcf332788db6ff2a', 'request_id': 'req-f5822f92-34a8-4e2b-b303-d30b4bccd6a9'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:09.590451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-f5822f92-34a8-4e2b-b303-d30b4bccd6a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f0f10f56afc411abcf332788db6ff2a', 'request_id': 'req-f5822f92-34a8-4e2b-b303-d30b4bccd6a9'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:09.612413 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-f5822f92-34a8-4e2b-b303-d30b4bccd6a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f0f10f56afc411abcf332788db6ff2a', 'request_id': 'req-f5822f92-34a8-4e2b-b303-d30b4bccd6a9'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:09.650252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-f5822f92-34a8-4e2b-b303-d30b4bccd6a9 admin admin] The token cache is being invalidate because project 0f0f10f56afc411abcf332788db6ff2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:09.650252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-f5822f92-34a8-4e2b-b303-d30b4bccd6a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:09.650641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-f5822f92-34a8-4e2b-b303-d30b4bccd6a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:09.654027 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 859/1698] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:09 2022] DELETE /identity/v3/projects/0f0f10f56afc411abcf332788db6ff2a => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:09.658687 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281636b-e85b-4763-8b45-da81ef54188c service swift] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:09.690747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5822f92-34a8-4e2b-b303-d30b4bccd6a9 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:09.701528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281636b-e85b-4763-8b45-da81ef54188c service swift] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:09.702026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f5822f92-34a8-4e2b-b303-d30b4bccd6a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:10.006366 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c1a17-c367-4e78-a1e4-f17fe1317407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:10.008355 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c1a17-c367-4e78-a1e4-f17fe1317407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:10.013243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c1a17-c367-4e78-a1e4-f17fe1317407 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:10.014090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c1a17-c367-4e78-a1e4-f17fe1317407 admin admin] PATH_INFO: `/v3/users/6e29b9588d4a4ac092acda9dcbaa67bf` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:10.017038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c1a17-c367-4e78-a1e4-f17fe1317407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e29b9588d4a4ac092acda9dcbaa67bf)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:10.032802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c1a17-c367-4e78-a1e4-f17fe1317407 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:10.053777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5822f92-34a8-4e2b-b303-d30b4bccd6a9 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:10.085597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15efce-0a23-47c1-be66-3a78bd9931b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:10.105078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15efce-0a23-47c1-be66-3a78bd9931b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:10.105456 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15efce-0a23-47c1-be66-3a78bd9931b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:10.105823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15efce-0a23-47c1-be66-3a78bd9931b8 admin admin] PATH_INFO: `/v3/users/1efcd6a2ce9e4a2ea7db5f9c0e21bbaa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:10.118542 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15efce-0a23-47c1-be66-3a78bd9931b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1efcd6a2ce9e4a2ea7db5f9c0e21bbaa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:10.132309 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15efce-0a23-47c1-be66-3a78bd9931b8 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:10.264206 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:10.270372 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-541c1a17-c367-4e78-a1e4-f17fe1317407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e29b9588d4a4ac092acda9dcbaa67bf'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:10.296392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4c15efce-0a23-47c1-be66-3a78bd9931b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1efcd6a2ce9e4a2ea7db5f9c0e21bbaa'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:10.327349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4c15efce-0a23-47c1-be66-3a78bd9931b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1efcd6a2ce9e4a2ea7db5f9c0e21bbaa'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:10.329383 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:10.330770 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4c15efce-0a23-47c1-be66-3a78bd9931b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1efcd6a2ce9e4a2ea7db5f9c0e21bbaa'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:10.352280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-541c1a17-c367-4e78-a1e4-f17fe1317407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e29b9588d4a4ac092acda9dcbaa67bf'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:10.353384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4c15efce-0a23-47c1-be66-3a78bd9931b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1efcd6a2ce9e4a2ea7db5f9c0e21bbaa'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:10.378060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4c15efce-0a23-47c1-be66-3a78bd9931b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1efcd6a2ce9e4a2ea7db5f9c0e21bbaa'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:10.389805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 860/1699] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:09 2022] DELETE /identity/v3/users/1efcd6a2ce9e4a2ea7db5f9c0e21bbaa => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:10.393726 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-541c1a17-c367-4e78-a1e4-f17fe1317407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e29b9588d4a4ac092acda9dcbaa67bf'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:10.394403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-541c1a17-c367-4e78-a1e4-f17fe1317407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e29b9588d4a4ac092acda9dcbaa67bf'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:10.394875 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b29dc1-fe30-4102-b684-b64020e03765 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:10.395025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-541c1a17-c367-4e78-a1e4-f17fe1317407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e29b9588d4a4ac092acda9dcbaa67bf'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:10.395477 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b29dc1-fe30-4102-b684-b64020e03765 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:10.395902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b29dc1-fe30-4102-b684-b64020e03765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:10.432101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-87b29dc1-fe30-4102-b684-b64020e03765 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:10.432369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-87b29dc1-fe30-4102-b684-b64020e03765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:10.475440 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 840/1700] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:09 2022] DELETE /identity/v3/users/6e29b9588d4a4ac092acda9dcbaa67bf => generated 0 bytes in 818 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:10.643280 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 861/1701] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:10.700054 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c1a17-c367-4e78-a1e4-f17fe1317407 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:10.702111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-541c1a17-c367-4e78-a1e4-f17fe1317407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:10.908032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c1a17-c367-4e78-a1e4-f17fe1317407 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:10.927116 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94070d49-9f68-4838-a510-77001ff544ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:10.927928 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94070d49-9f68-4838-a510-77001ff544ad service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:10.928080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94070d49-9f68-4838-a510-77001ff544ad service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:10.928526 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94070d49-9f68-4838-a510-77001ff544ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:11.160659 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94070d49-9f68-4838-a510-77001ff544ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:11.168425 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94070d49-9f68-4838-a510-77001ff544ad service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:11.224653 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 841/1702] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:11.247169 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7af615-2f9f-4d25-92d5-6a4a8cf42e57 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:11.247745 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7af615-2f9f-4d25-92d5-6a4a8cf42e57 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:11.248225 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7af615-2f9f-4d25-92d5-6a4a8cf42e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:11.317752 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-cb7af615-2f9f-4d25-92d5-6a4a8cf42e57 None None] MFA Rules not processed for user `42dee639e2834d99b7468f911c37cf8f`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:11.326085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-cb7af615-2f9f-4d25-92d5-6a4a8cf42e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:11.349827 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 862/1703] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Jun 27 15:48:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:11.401036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94070d49-9f68-4838-a510-77001ff544ad service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:11.422485 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94070d49-9f68-4838-a510-77001ff544ad service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:11.434331 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333df938-3880-4585-abf2-4c8604453963 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:11.435020 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333df938-3880-4585-abf2-4c8604453963 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:11.435352 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333df938-3880-4585-abf2-4c8604453963 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:11.435645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333df938-3880-4585-abf2-4c8604453963 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:11.446676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333df938-3880-4585-abf2-4c8604453963 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:11.448532 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333df938-3880-4585-abf2-4c8604453963 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:11.487197 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 842/1704] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:11.956878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb3078-5a57-46f7-acdd-ba64bbd8d5d7 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:11.957181 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb3078-5a57-46f7-acdd-ba64bbd8d5d7 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:11.957834 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb3078-5a57-46f7-acdd-ba64bbd8d5d7 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:11.959819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 863/1705] 10.222.0.20 () {54 vars in 912 bytes} [Mon Jun 27 15:48:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:48:11.972133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2e216-80b5-4f61-b7b4-ca6d57614e89 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:11.974062 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2e216-80b5-4f61-b7b4-ca6d57614e89 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:11.974591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2e216-80b5-4f61-b7b4-ca6d57614e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:12.050981 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-46e2e216-80b5-4f61-b7b4-ca6d57614e89 None None] MFA Rules not processed for user `42dee639e2834d99b7468f911c37cf8f`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:12.058689 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-46e2e216-80b5-4f61-b7b4-ca6d57614e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:12.102107 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 843/1706] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jun 27 15:48:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:48:12.267988 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb3078-5a57-46f7-acdd-ba64bbd8d5d7 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:12.270631 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-87eb3078-5a57-46f7-acdd-ba64bbd8d5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:12.306293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb3078-5a57-46f7-acdd-ba64bbd8d5d7 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:12.321276 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562f089-2b79-43d1-b912-67a179f238d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:12.322028 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562f089-2b79-43d1-b912-67a179f238d7 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:12.322475 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562f089-2b79-43d1-b912-67a179f238d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:12.322763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562f089-2b79-43d1-b912-67a179f238d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:12.354663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562f089-2b79-43d1-b912-67a179f238d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:12.356102 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562f089-2b79-43d1-b912-67a179f238d7 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:12.392460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 864/1707] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:12.878490 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2e216-80b5-4f61-b7b4-ca6d57614e89 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:12.880095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-46e2e216-80b5-4f61-b7b4-ca6d57614e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:12.915268 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2e216-80b5-4f61-b7b4-ca6d57614e89 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:12.932486 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc1f15-f059-448d-bfa9-d78db3790381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:12.937919 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc1f15-f059-448d-bfa9-d78db3790381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:12.941206 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc1f15-f059-448d-bfa9-d78db3790381 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:12.941730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc1f15-f059-448d-bfa9-d78db3790381 admin admin] PATH_INFO: `/v3/projects/4b12bd7199c94e6392710d44c8b8c24e` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:12.947821 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc1f15-f059-448d-bfa9-d78db3790381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b12bd7199c94e6392710d44c8b8c24e)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:12.952335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc1f15-f059-448d-bfa9-d78db3790381 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:13.081004 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2fbc1f15-f059-448d-bfa9-d78db3790381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b12bd7199c94e6392710d44c8b8c24e', 'request_id': 'req-2fbc1f15-f059-448d-bfa9-d78db3790381'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:13.106594 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2fbc1f15-f059-448d-bfa9-d78db3790381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b12bd7199c94e6392710d44c8b8c24e', 'request_id': 'req-2fbc1f15-f059-448d-bfa9-d78db3790381'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:13.128776 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:13.133858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2fbc1f15-f059-448d-bfa9-d78db3790381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b12bd7199c94e6392710d44c8b8c24e', 'request_id': 'req-2fbc1f15-f059-448d-bfa9-d78db3790381'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:13.177890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2fbc1f15-f059-448d-bfa9-d78db3790381 admin admin] The token cache is being invalidate because project 4b12bd7199c94e6392710d44c8b8c24e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:13.179628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2fbc1f15-f059-448d-bfa9-d78db3790381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:13.182579 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2fbc1f15-f059-448d-bfa9-d78db3790381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:13.187919 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 844/1708] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:12 2022] DELETE /identity/v3/projects/4b12bd7199c94e6392710d44c8b8c24e => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:13.192798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ef49a-eb6d-4375-b9f6-ac5395d1cee0 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:13.194542 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ef49a-eb6d-4375-b9f6-ac5395d1cee0 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:13.195396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ef49a-eb6d-4375-b9f6-ac5395d1cee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:13.257534 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-250ef49a-eb6d-4375-b9f6-ac5395d1cee0 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:13.260550 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-250ef49a-eb6d-4375-b9f6-ac5395d1cee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:13.482905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 865/1709] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:13.490922 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d2d49-f53e-473f-af3b-f69be78e3667 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:13.491248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d2d49-f53e-473f-af3b-f69be78e3667 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:13.491547 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d2d49-f53e-473f-af3b-f69be78e3667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:13.557265 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-4f4d2d49-f53e-473f-af3b-f69be78e3667 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:13.558712 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4f4d2d49-f53e-473f-af3b-f69be78e3667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:13.595326 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 845/1710] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:13.599497 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ef49a-eb6d-4375-b9f6-ac5395d1cee0 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:13.619584 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ef49a-eb6d-4375-b9f6-ac5395d1cee0 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:13.637981 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64e5f4-ef5f-47f2-bd66-7f7ad180ca71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:13.641982 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64e5f4-ef5f-47f2-bd66-7f7ad180ca71 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:13.642280 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64e5f4-ef5f-47f2-bd66-7f7ad180ca71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:13.647582 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64e5f4-ef5f-47f2-bd66-7f7ad180ca71 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:13.648615 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64e5f4-ef5f-47f2-bd66-7f7ad180ca71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:48:13.649041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64e5f4-ef5f-47f2-bd66-7f7ad180ca71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:13.674061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a64e5f4-ef5f-47f2-bd66-7f7ad180ca71 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:13.758910 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 866/1711] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:48:13.763486 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d2d49-f53e-473f-af3b-f69be78e3667 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:13.784086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d2d49-f53e-473f-af3b-f69be78e3667 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:13.830102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef1a43-271d-4881-b9a1-4ce78d51578a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:13.830890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef1a43-271d-4881-b9a1-4ce78d51578a admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:13.831219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef1a43-271d-4881-b9a1-4ce78d51578a admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:13.831499 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef1a43-271d-4881-b9a1-4ce78d51578a admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:13.832288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef1a43-271d-4881-b9a1-4ce78d51578a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:13.854958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef1a43-271d-4881-b9a1-4ce78d51578a admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:13.945000 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 846/1712] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:13 2022] POST /identity/v3/projects => generated 405 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:13.949143 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64e5f4-ef5f-47f2-bd66-7f7ad180ca71 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:13.965090 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a64e5f4-ef5f-47f2-bd66-7f7ad180ca71 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:14.029565 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:14.272745 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef1a43-271d-4881-b9a1-4ce78d51578a admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:14.275539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-ebef1a43-271d-4881-b9a1-4ce78d51578a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:14.275961 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955844b-c472-46b9-9c07-ee40eb2aa927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:14.282585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955844b-c472-46b9-9c07-ee40eb2aa927 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:14.282876 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955844b-c472-46b9-9c07-ee40eb2aa927 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:14.283675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955844b-c472-46b9-9c07-ee40eb2aa927 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:14.287724 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955844b-c472-46b9-9c07-ee40eb2aa927 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:14.291650 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955844b-c472-46b9-9c07-ee40eb2aa927 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:14.306107 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef1a43-271d-4881-b9a1-4ce78d51578a admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:14.342714 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae06ada-6146-426c-9bc9-7dcd42be3913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:14.345474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae06ada-6146-426c-9bc9-7dcd42be3913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:14.347509 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae06ada-6146-426c-9bc9-7dcd42be3913 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:14.349823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae06ada-6146-426c-9bc9-7dcd42be3913 admin admin] PATH_INFO: `/v3/projects/8305be07ffd7400bb67e5e9c52668a35` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:14.353948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae06ada-6146-426c-9bc9-7dcd42be3913 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8305be07ffd7400bb67e5e9c52668a35)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:14.391516 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae06ada-6146-426c-9bc9-7dcd42be3913 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:14.394637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 867/1713] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:13 2022] POST /identity/v3/users => generated 303 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:14.400983 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955844b-c472-46b9-9c07-ee40eb2aa927 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:14.456508 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955844b-c472-46b9-9c07-ee40eb2aa927 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:14.478175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9dab2-598f-474e-9759-f8d59069f426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:14.480414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9dab2-598f-474e-9759-f8d59069f426 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:14.480902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9dab2-598f-474e-9759-f8d59069f426 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:14.481342 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9dab2-598f-474e-9759-f8d59069f426 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:14.485232 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9dab2-598f-474e-9759-f8d59069f426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:14.491721 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9dab2-598f-474e-9759-f8d59069f426 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:14.572579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 868/1714] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:14.578408 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9dab2-598f-474e-9759-f8d59069f426 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:14.619137 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9dab2-598f-474e-9759-f8d59069f426 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:14.664231 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-cae06ada-6146-426c-9bc9-7dcd42be3913 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8305be07ffd7400bb67e5e9c52668a35', 'request_id': 'req-cae06ada-6146-426c-9bc9-7dcd42be3913'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:14.691569 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-cae06ada-6146-426c-9bc9-7dcd42be3913 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8305be07ffd7400bb67e5e9c52668a35', 'request_id': 'req-cae06ada-6146-426c-9bc9-7dcd42be3913'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:14.701392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-cae06ada-6146-426c-9bc9-7dcd42be3913 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8305be07ffd7400bb67e5e9c52668a35', 'request_id': 'req-cae06ada-6146-426c-9bc9-7dcd42be3913'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:14.733230 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-cae06ada-6146-426c-9bc9-7dcd42be3913 admin admin] The token cache is being invalidate because project 8305be07ffd7400bb67e5e9c52668a35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:14.734373 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-cae06ada-6146-426c-9bc9-7dcd42be3913 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:14.739409 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-cae06ada-6146-426c-9bc9-7dcd42be3913 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:14.745709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 847/1715] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:14 2022] DELETE /identity/v3/projects/8305be07ffd7400bb67e5e9c52668a35 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:48:14.751853 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2987fe8-751d-4197-aa0c-5ed234908ce6 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:14.754861 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2987fe8-751d-4197-aa0c-5ed234908ce6 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:14.758196 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2987fe8-751d-4197-aa0c-5ed234908ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:14.869921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-a2987fe8-751d-4197-aa0c-5ed234908ce6 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:14.874001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a2987fe8-751d-4197-aa0c-5ed234908ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:14.881923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95ba79-82ca-4346-9573-c7ba81d013aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:14.885970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95ba79-82ca-4346-9573-c7ba81d013aa admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:14.886265 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95ba79-82ca-4346-9573-c7ba81d013aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:14.886554 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95ba79-82ca-4346-9573-c7ba81d013aa admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:14.887333 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95ba79-82ca-4346-9573-c7ba81d013aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:14.895841 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95ba79-82ca-4346-9573-c7ba81d013aa admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:14.905993 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 848/1716] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:14.915232 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773f86c-5c57-4852-b0aa-5ed7202e9ea4 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:14.915688 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773f86c-5c57-4852-b0aa-5ed7202e9ea4 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:14.916081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773f86c-5c57-4852-b0aa-5ed7202e9ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:14.935266 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 869/1717] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:14.939001 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95ba79-82ca-4346-9573-c7ba81d013aa admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:14.946404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-ee95ba79-82ca-4346-9573-c7ba81d013aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:14.968028 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-2773f86c-5c57-4852-b0aa-5ed7202e9ea4 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:14.970080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-2773f86c-5c57-4852-b0aa-5ed7202e9ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:15.024237 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95ba79-82ca-4346-9573-c7ba81d013aa admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:15.034110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ab0e7-630b-43c8-b8b0-2fa899888c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:15.035918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ab0e7-630b-43c8-b8b0-2fa899888c10 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:15.036766 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ab0e7-630b-43c8-b8b0-2fa899888c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:15.037457 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ab0e7-630b-43c8-b8b0-2fa899888c10 admin admin] PATH_INFO: `/v3/projects/7dbc67b63ef442bc9731b67b8f2eff96/users/d2ecf39af75b4ac29b9a65312fd3cf12/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:15.045483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 849/1718] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:48:15.062009 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773f86c-5c57-4852-b0aa-5ed7202e9ea4 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:15.092390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ab0e7-630b-43c8-b8b0-2fa899888c10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dbc67b63ef442bc9731b67b8f2eff96, user_id=d2ecf39af75b4ac29b9a65312fd3cf12, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:15.098042 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ab0e7-630b-43c8-b8b0-2fa899888c10 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:15.118579 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773f86c-5c57-4852-b0aa-5ed7202e9ea4 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:15.140214 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54819582-a722-4193-a0ae-500fa2345402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:15.143846 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54819582-a722-4193-a0ae-500fa2345402 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:15.146399 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54819582-a722-4193-a0ae-500fa2345402 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:15.149558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54819582-a722-4193-a0ae-500fa2345402 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:15.151949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54819582-a722-4193-a0ae-500fa2345402 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:48:15.152409 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54819582-a722-4193-a0ae-500fa2345402 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:15.155118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54819582-a722-4193-a0ae-500fa2345402 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:15.168477 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 870/1719] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:14 2022] PUT /identity/v3/projects/7dbc67b63ef442bc9731b67b8f2eff96/users/d2ecf39af75b4ac29b9a65312fd3cf12/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:15.176314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c5bf0-20a3-4997-9897-07fa432a58f4 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:15.176314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c5bf0-20a3-4997-9897-07fa432a58f4 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:15.176314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c5bf0-20a3-4997-9897-07fa432a58f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:15.197259 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 850/1720] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:48:15.202433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54819582-a722-4193-a0ae-500fa2345402 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:15.219793 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54819582-a722-4193-a0ae-500fa2345402 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:15.238667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-a89c5bf0-20a3-4997-9897-07fa432a58f4 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:15.249834 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-a89c5bf0-20a3-4997-9897-07fa432a58f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:15.442433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496157a-a129-44c8-b808-b1deb5268eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:15.443137 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496157a-a129-44c8-b808-b1deb5268eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:15.443791 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496157a-a129-44c8-b808-b1deb5268eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:15.443791 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496157a-a129-44c8-b808-b1deb5268eb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:15.444603 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496157a-a129-44c8-b808-b1deb5268eb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:15.446440 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b496157a-a129-44c8-b808-b1deb5268eb0 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:15.471034 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 871/1721] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:15.489263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b8e66-8d4e-498e-9d7f-d35873d4957e None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:15.489961 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b8e66-8d4e-498e-9d7f-d35873d4957e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:15.489961 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b8e66-8d4e-498e-9d7f-d35873d4957e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:15.542642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-434b8e66-8d4e-498e-9d7f-d35873d4957e None None] MFA Rules not processed for user `d35c652539f443d4895b5f81461833ba`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:15.547885 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 851/1722] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:15 2022] POST /identity/v3/projects => generated 377 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:15.548060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-434b8e66-8d4e-498e-9d7f-d35873d4957e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:15.552441 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496157a-a129-44c8-b808-b1deb5268eb0 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:15.563988 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:15.566518 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:15.874420 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 872/1723] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:48:15.874420 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b8e66-8d4e-498e-9d7f-d35873d4957e None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:15.880115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-434b8e66-8d4e-498e-9d7f-d35873d4957e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:15.882237 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496157a-a129-44c8-b808-b1deb5268eb0 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:15.895013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d075c-2b43-45db-a7bb-c72310f537e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:15.895856 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d075c-2b43-45db-a7bb-c72310f537e9 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:15.897488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d075c-2b43-45db-a7bb-c72310f537e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:15.898778 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d075c-2b43-45db-a7bb-c72310f537e9 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:15.900901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d075c-2b43-45db-a7bb-c72310f537e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:15.903132 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d075c-2b43-45db-a7bb-c72310f537e9 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:15.966561 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 852/1724] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:15 2022] POST /identity/v3/users => generated 289 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:15.969092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f13a4a-1be2-42a1-8fe8-41f562cb2044 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:15.969605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f13a4a-1be2-42a1-8fe8-41f562cb2044 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:15.970022 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f13a4a-1be2-42a1-8fe8-41f562cb2044 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:15.972569 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 853/1725] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jun 27 15:48:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:48:15.973455 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f13a4a-1be2-42a1-8fe8-41f562cb2044 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:15.978647 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-69f13a4a-1be2-42a1-8fe8-41f562cb2044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:16.011395 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f13a4a-1be2-42a1-8fe8-41f562cb2044 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:16.053926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca290611-85dd-43ec-b41d-32f171a83adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:16.058467 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca290611-85dd-43ec-b41d-32f171a83adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:16.059961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca290611-85dd-43ec-b41d-32f171a83adf admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:16.063008 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca290611-85dd-43ec-b41d-32f171a83adf admin admin] PATH_INFO: `/v3/projects/ce922fd498c84029a0123e1866b31dc5` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:16.066616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca290611-85dd-43ec-b41d-32f171a83adf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce922fd498c84029a0123e1866b31dc5)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:16.071380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca290611-85dd-43ec-b41d-32f171a83adf admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:16.085843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b8e66-8d4e-498e-9d7f-d35873d4957e None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:16.094448 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce1571-7311-4a7f-bcf5-69438cc6c6f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:16.095073 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce1571-7311-4a7f-bcf5-69438cc6c6f4 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:16.095363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce1571-7311-4a7f-bcf5-69438cc6c6f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:16.096933 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce1571-7311-4a7f-bcf5-69438cc6c6f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:16.145137 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce1571-7311-4a7f-bcf5-69438cc6c6f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:16.158746 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce1571-7311-4a7f-bcf5-69438cc6c6f4 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:16.206644 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 873/1726] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:16.210992 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce1571-7311-4a7f-bcf5-69438cc6c6f4 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:16.256418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ca290611-85dd-43ec-b41d-32f171a83adf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce922fd498c84029a0123e1866b31dc5', 'request_id': 'req-ca290611-85dd-43ec-b41d-32f171a83adf'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:16.290712 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ca290611-85dd-43ec-b41d-32f171a83adf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce922fd498c84029a0123e1866b31dc5', 'request_id': 'req-ca290611-85dd-43ec-b41d-32f171a83adf'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:16.308660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce1571-7311-4a7f-bcf5-69438cc6c6f4 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:16.310781 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ca290611-85dd-43ec-b41d-32f171a83adf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce922fd498c84029a0123e1866b31dc5', 'request_id': 'req-ca290611-85dd-43ec-b41d-32f171a83adf'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:16.340534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ca290611-85dd-43ec-b41d-32f171a83adf admin admin] The token cache is being invalidate because project ce922fd498c84029a0123e1866b31dc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:16.341317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ca290611-85dd-43ec-b41d-32f171a83adf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:16.345598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ca290611-85dd-43ec-b41d-32f171a83adf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:16.354380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 854/1727] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:15 2022] DELETE /identity/v3/projects/ce922fd498c84029a0123e1866b31dc5 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:16.356469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c420258-e26a-4c2e-9961-a533769f8272 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:16.360929 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c420258-e26a-4c2e-9961-a533769f8272 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:16.361307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c420258-e26a-4c2e-9961-a533769f8272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:16.429204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-9c420258-e26a-4c2e-9961-a533769f8272 None None] MFA Rules not processed for user `1c8f6262cf24419daf803ae71aed24f5`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:16.433512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-9c420258-e26a-4c2e-9961-a533769f8272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:16.656639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6b3e8-0b6a-4618-81f3-cd0febc7b192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:16.661182 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6b3e8-0b6a-4618-81f3-cd0febc7b192 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:16.661780 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6b3e8-0b6a-4618-81f3-cd0febc7b192 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:16.662643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6b3e8-0b6a-4618-81f3-cd0febc7b192 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:16.665059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c6b3e8-0b6a-4618-81f3-cd0febc7b192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:16.670493 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c6b3e8-0b6a-4618-81f3-cd0febc7b192 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:16.705297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 874/1728] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 493 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:16.707977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e36be8-2d60-4d95-94ec-7b671dd27b88 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:16.710437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e36be8-2d60-4d95-94ec-7b671dd27b88 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:16.712148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e36be8-2d60-4d95-94ec-7b671dd27b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:16.713187 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 855/1729] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jun 27 15:48:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:16.715409 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c420258-e26a-4c2e-9961-a533769f8272 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:16.721800 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-9c420258-e26a-4c2e-9961-a533769f8272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:16.766696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c420258-e26a-4c2e-9961-a533769f8272 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:16.781921 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-41e36be8-2d60-4d95-94ec-7b671dd27b88 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:16.782146 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913f4fc-4710-41a7-919f-7618b7be6902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:16.784219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913f4fc-4710-41a7-919f-7618b7be6902 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:16.785384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913f4fc-4710-41a7-919f-7618b7be6902 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:16.786068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913f4fc-4710-41a7-919f-7618b7be6902 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:16.788363 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913f4fc-4710-41a7-919f-7618b7be6902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:16.792108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-41e36be8-2d60-4d95-94ec-7b671dd27b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:16.797174 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913f4fc-4710-41a7-919f-7618b7be6902 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:16.806849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:16.853347 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 856/1730] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:16.854396 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913f4fc-4710-41a7-919f-7618b7be6902 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:16.871653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 875/1731] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:16.895196 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913f4fc-4710-41a7-919f-7618b7be6902 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:16.913547 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e8c64-5c2e-4f1c-adf8-ab541c54acd7 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:16.914119 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e8c64-5c2e-4f1c-adf8-ab541c54acd7 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:16.914638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e8c64-5c2e-4f1c-adf8-ab541c54acd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:16.920241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8918d-0037-4210-a7d8-0dbc1e90dfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:16.922829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8918d-0037-4210-a7d8-0dbc1e90dfd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:16.937981 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8918d-0037-4210-a7d8-0dbc1e90dfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:16.937981 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8918d-0037-4210-a7d8-0dbc1e90dfd8 admin admin] PATH_INFO: `/v3/projects/2dfc884612e447e8abe3c44036cf8831/users/4e86862914304673a384443dc6713650/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:17.000900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8918d-0037-4210-a7d8-0dbc1e90dfd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dfc884612e447e8abe3c44036cf8831, user_id=4e86862914304673a384443dc6713650, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:17.003753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8918d-0037-4210-a7d8-0dbc1e90dfd8 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:17.078184 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 857/1732] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:16 2022] PUT /identity/v3/projects/2dfc884612e447e8abe3c44036cf8831/users/4e86862914304673a384443dc6713650/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:17.096801 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b5f54-a1d4-4438-8b85-705d3c2f34bb None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:17.097418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b5f54-a1d4-4438-8b85-705d3c2f34bb None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:17.099171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b5f54-a1d4-4438-8b85-705d3c2f34bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:17.152991 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-c45e8c64-5c2e-4f1c-adf8-ab541c54acd7 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:17.154638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-c45e8c64-5c2e-4f1c-adf8-ab541c54acd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:17.204087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-c77b5f54-a1d4-4438-8b85-705d3c2f34bb None None] MFA Rules not processed for user `4e86862914304673a384443dc6713650`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:17.205486 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-c77b5f54-a1d4-4438-8b85-705d3c2f34bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:17.377738 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:17.495430 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 876/1733] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 590 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:17.511955 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8e0c1-b7d4-4c44-b4c2-4ee460f4c4f6 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:17.518804 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8e0c1-b7d4-4c44-b4c2-4ee460f4c4f6 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:17.518994 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8e0c1-b7d4-4c44-b4c2-4ee460f4c4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:17.646445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 858/1734] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 551 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:17.647942 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b5f54-a1d4-4438-8b85-705d3c2f34bb None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:17.656058 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-00c8e0c1-b7d4-4c44-b4c2-4ee460f4c4f6 None None] MFA Rules not processed for user `1a7a81da8acf40849a9ea86e2542ae6a`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:17.664473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-00c8e0c1-b7d4-4c44-b4c2-4ee460f4c4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:17.687613 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b5f54-a1d4-4438-8b85-705d3c2f34bb None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:17.721128 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1e6e6-85cb-4775-9903-2fabb5221dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:17.721128 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1e6e6-85cb-4775-9903-2fabb5221dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:17.736903 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1e6e6-85cb-4775-9903-2fabb5221dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:17.737413 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1e6e6-85cb-4775-9903-2fabb5221dd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:17.744484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1e6e6-85cb-4775-9903-2fabb5221dd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:48:17.745406 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1e6e6-85cb-4775-9903-2fabb5221dd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:17.746972 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d1e6e6-85cb-4775-9903-2fabb5221dd8 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:17.863123 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 859/1735] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:48:17.870490 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1e6e6-85cb-4775-9903-2fabb5221dd8 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:17.923386 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1e6e6-85cb-4775-9903-2fabb5221dd8 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:17.948269 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672123d8-e2b4-4043-97ce-3697913aedb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:17.956640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672123d8-e2b4-4043-97ce-3697913aedb5 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:17.957942 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672123d8-e2b4-4043-97ce-3697913aedb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:17.957942 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672123d8-e2b4-4043-97ce-3697913aedb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:17.958372 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672123d8-e2b4-4043-97ce-3697913aedb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:17.969856 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 877/1736] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:17.971453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8e0c1-b7d4-4c44-b4c2-4ee460f4c4f6 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:17.974263 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672123d8-e2b4-4043-97ce-3697913aedb5 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:17.998088 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8e0c1-b7d4-4c44-b4c2-4ee460f4c4f6 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:18.021873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f9dcf-350f-4a86-be22-157a46cbecca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:18.029210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f9dcf-350f-4a86-be22-157a46cbecca admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:18.030028 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f9dcf-350f-4a86-be22-157a46cbecca admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:18.034613 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f9dcf-350f-4a86-be22-157a46cbecca admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:18.036767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f9dcf-350f-4a86-be22-157a46cbecca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:18.043097 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f9dcf-350f-4a86-be22-157a46cbecca admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:18.142025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 860/1737] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:17 2022] POST /identity/v3/projects => generated 375 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:18.143743 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672123d8-e2b4-4043-97ce-3697913aedb5 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:18.228464 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 878/1738] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:17 2022] POST /identity/v3/projects => generated 363 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:18.229498 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f9dcf-350f-4a86-be22-157a46cbecca admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:18.405421 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672123d8-e2b4-4043-97ce-3697913aedb5 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:18.484194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f9dcf-350f-4a86-be22-157a46cbecca admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:18.488085 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:18.501539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3557e7d-0477-44f6-8ad9-aead2c87624c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:18.504254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3557e7d-0477-44f6-8ad9-aead2c87624c admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:18.508725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3557e7d-0477-44f6-8ad9-aead2c87624c admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:18.510064 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3557e7d-0477-44f6-8ad9-aead2c87624c admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:18.511621 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3557e7d-0477-44f6-8ad9-aead2c87624c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:18.519683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3557e7d-0477-44f6-8ad9-aead2c87624c admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:18.559306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f015b92-d887-48ff-9041-ce1291e525ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:18.560021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f015b92-d887-48ff-9041-ce1291e525ad admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:18.572448 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 879/1739] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:18 2022] POST /identity/v3/users => generated 282 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:18.578616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f015b92-d887-48ff-9041-ce1291e525ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:18.579013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f015b92-d887-48ff-9041-ce1291e525ad admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:18.579909 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3557e7d-0477-44f6-8ad9-aead2c87624c admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:18.583527 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f015b92-d887-48ff-9041-ce1291e525ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:18.586458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f015b92-d887-48ff-9041-ce1291e525ad admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:18.604490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3557e7d-0477-44f6-8ad9-aead2c87624c admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:18.623481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff4627a-e5d8-44b4-b5a6-de16a2dbef96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:18.626352 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4627a-e5d8-44b4-b5a6-de16a2dbef96 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:18.628188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4627a-e5d8-44b4-b5a6-de16a2dbef96 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:18.628471 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4627a-e5d8-44b4-b5a6-de16a2dbef96 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:18.629571 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff4627a-e5d8-44b4-b5a6-de16a2dbef96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:18.638456 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff4627a-e5d8-44b4-b5a6-de16a2dbef96 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:18.643668 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 861/1740] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:18 2022] POST /identity/v3/users => generated 294 bytes in 501 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:18.647865 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f015b92-d887-48ff-9041-ce1291e525ad admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:18.672273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f015b92-d887-48ff-9041-ce1291e525ad admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:18.679359 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 880/1741] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:18.680385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff4627a-e5d8-44b4-b5a6-de16a2dbef96 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:18.690071 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a0962-3bfb-4f37-9414-e420bd6b10aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:18.690952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a0962-3bfb-4f37-9414-e420bd6b10aa admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:18.691247 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a0962-3bfb-4f37-9414-e420bd6b10aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:18.691526 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a0962-3bfb-4f37-9414-e420bd6b10aa admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:18.692274 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a0962-3bfb-4f37-9414-e420bd6b10aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:18.694226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a0962-3bfb-4f37-9414-e420bd6b10aa admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:18.712746 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff4627a-e5d8-44b4-b5a6-de16a2dbef96 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:18.717598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 862/1742] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:48:18.722111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a0962-3bfb-4f37-9414-e420bd6b10aa admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:18.726138 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f9e55-cad7-4ec4-8372-2dbb40fbe1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:18.729201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f9e55-cad7-4ec4-8372-2dbb40fbe1d9 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:18.729528 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f9e55-cad7-4ec4-8372-2dbb40fbe1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:18.729846 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f9e55-cad7-4ec4-8372-2dbb40fbe1d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:18.730621 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f9e55-cad7-4ec4-8372-2dbb40fbe1d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:18.737689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f9e55-cad7-4ec4-8372-2dbb40fbe1d9 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:18.741809 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a0962-3bfb-4f37-9414-e420bd6b10aa admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:18.762340 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87943fe8-7923-42d5-9d36-a1302df7120f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:18.763684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87943fe8-7923-42d5-9d36-a1302df7120f admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:18.763995 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87943fe8-7923-42d5-9d36-a1302df7120f admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:18.764446 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87943fe8-7923-42d5-9d36-a1302df7120f admin admin] PATH_INFO: `/v3/projects/c3f28feb32b04215b6dc42ed33afa582/users/a760404a55804f659c2a3c26e7266bde/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:18.774532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 881/1743] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:18.790231 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f9e55-cad7-4ec4-8372-2dbb40fbe1d9 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:18.800304 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87943fe8-7923-42d5-9d36-a1302df7120f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f28feb32b04215b6dc42ed33afa582, user_id=a760404a55804f659c2a3c26e7266bde, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:18.807053 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87943fe8-7923-42d5-9d36-a1302df7120f admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:18.848602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 863/1744] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:18 2022] PUT /identity/v3/projects/c3f28feb32b04215b6dc42ed33afa582/users/a760404a55804f659c2a3c26e7266bde/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:48:18.852822 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f9e55-cad7-4ec4-8372-2dbb40fbe1d9 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:18.853732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87943fe8-7923-42d5-9d36-a1302df7120f admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:18.970310 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:19.015764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6d912-b313-468d-960f-e80f0520d8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:19.017200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6d912-b313-468d-960f-e80f0520d8f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:19.017494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6d912-b313-468d-960f-e80f0520d8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:19.017810 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6d912-b313-468d-960f-e80f0520d8f7 admin admin] PATH_INFO: `/v3/projects/f9983cf7d78d470887d346186b71b938/users/898af1d0e74f415c8555d5e4a35e0232/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:19.035021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87943fe8-7923-42d5-9d36-a1302df7120f admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:19.060600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6d912-b313-468d-960f-e80f0520d8f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9983cf7d78d470887d346186b71b938, user_id=898af1d0e74f415c8555d5e4a35e0232, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:19.061271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538db19-8fd5-405e-8e31-35e57b81f17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:19.062297 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538db19-8fd5-405e-8e31-35e57b81f17d admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:19.062708 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538db19-8fd5-405e-8e31-35e57b81f17d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:19.063055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538db19-8fd5-405e-8e31-35e57b81f17d admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:19.063883 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538db19-8fd5-405e-8e31-35e57b81f17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:19.065718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6d912-b313-468d-960f-e80f0520d8f7 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:19.072870 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538db19-8fd5-405e-8e31-35e57b81f17d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:19.092260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 882/1745] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:18 2022] PUT /identity/v3/projects/f9983cf7d78d470887d346186b71b938/users/898af1d0e74f415c8555d5e4a35e0232/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:19.096008 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 864/1746] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:19.102958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6d912-b313-468d-960f-e80f0520d8f7 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:19.106619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17013fe-e22d-47ac-994a-0944a12cb58c None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:19.107202 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17013fe-e22d-47ac-994a-0944a12cb58c None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:19.107676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17013fe-e22d-47ac-994a-0944a12cb58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:19.216201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-f17013fe-e22d-47ac-994a-0944a12cb58c None None] MFA Rules not processed for user `898af1d0e74f415c8555d5e4a35e0232`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:19.218451 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f17013fe-e22d-47ac-994a-0944a12cb58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:19.476352 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6d912-b313-468d-960f-e80f0520d8f7 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:19.522473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7b9b3-4823-4e67-8350-80d83f4c9ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:19.528373 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7b9b3-4823-4e67-8350-80d83f4c9ca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:19.530680 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7b9b3-4823-4e67-8350-80d83f4c9ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:19.531788 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7b9b3-4823-4e67-8350-80d83f4c9ca9 admin admin] PATH_INFO: `/v3/projects/c3f28feb32b04215b6dc42ed33afa582/users/a760404a55804f659c2a3c26e7266bde/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:19.547057 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7b9b3-4823-4e67-8350-80d83f4c9ca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f28feb32b04215b6dc42ed33afa582, user_id=a760404a55804f659c2a3c26e7266bde, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:19.558325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7b9b3-4823-4e67-8350-80d83f4c9ca9 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:19.595069 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 865/1747] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:19 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 489 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:48:19.616633 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17013fe-e22d-47ac-994a-0944a12cb58c None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:19.616851 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 883/1748] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:19 2022] PUT /identity/v3/projects/c3f28feb32b04215b6dc42ed33afa582/users/a760404a55804f659c2a3c26e7266bde/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:19.617934 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7b9b3-4823-4e67-8350-80d83f4c9ca9 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:19.628700 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f17013fe-e22d-47ac-994a-0944a12cb58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:19.670716 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:19.866585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7b9b3-4823-4e67-8350-80d83f4c9ca9 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:19.893194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8206c6-fcba-4647-9672-35ae3c12fad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:19.894797 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8206c6-fcba-4647-9672-35ae3c12fad8 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:19.895070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8206c6-fcba-4647-9672-35ae3c12fad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:19.895361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8206c6-fcba-4647-9672-35ae3c12fad8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:19.896518 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8206c6-fcba-4647-9672-35ae3c12fad8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:19.920227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8206c6-fcba-4647-9672-35ae3c12fad8 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:19.934529 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17013fe-e22d-47ac-994a-0944a12cb58c None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:20.030038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 884/1749] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:19 2022] POST /identity/v3/projects => generated 365 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:20.032919 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388d262-72e6-453d-b49f-498c2a0cbee9 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:20.033496 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388d262-72e6-453d-b49f-498c2a0cbee9 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:20.033847 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388d262-72e6-453d-b49f-498c2a0cbee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:20.163550 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-8388d262-72e6-453d-b49f-498c2a0cbee9 None None] MFA Rules not processed for user `a760404a55804f659c2a3c26e7266bde`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:20.165086 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8388d262-72e6-453d-b49f-498c2a0cbee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:20.173330 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee6f61-e2e2-489a-a5f1-39355d784f5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:20.174106 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee6f61-e2e2-489a-a5f1-39355d784f5e service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:20.174394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee6f61-e2e2-489a-a5f1-39355d784f5e service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:20.174684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee6f61-e2e2-489a-a5f1-39355d784f5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:20.215336 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee6f61-e2e2-489a-a5f1-39355d784f5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:20.229731 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee6f61-e2e2-489a-a5f1-39355d784f5e service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:20.261959 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 866/1750] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:20.264510 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee6f61-e2e2-489a-a5f1-39355d784f5e service nova] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:20.290089 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:20.292947 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee6f61-e2e2-489a-a5f1-39355d784f5e service nova] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:20.574228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 885/1751] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 543 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:20.578276 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388d262-72e6-453d-b49f-498c2a0cbee9 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:20.581007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8388d262-72e6-453d-b49f-498c2a0cbee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:20.628121 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce045c-a952-46cb-8fdb-b66605e0c4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:20.630794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce045c-a952-46cb-8fdb-b66605e0c4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:20.636127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce045c-a952-46cb-8fdb-b66605e0c4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:20.636826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce045c-a952-46cb-8fdb-b66605e0c4b5 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:20.644218 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce045c-a952-46cb-8fdb-b66605e0c4b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:20.651352 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce045c-a952-46cb-8fdb-b66605e0c4b5 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:20.734975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 867/1752] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:20 2022] POST /identity/v3/users => generated 289 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:20.755260 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ef631-0876-41fe-a1bc-46f4a9499eca None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:20.755863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ef631-0876-41fe-a1bc-46f4a9499eca None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:20.761428 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ef631-0876-41fe-a1bc-46f4a9499eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:20.875687 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-1a0ef631-0876-41fe-a1bc-46f4a9499eca None None] MFA Rules not processed for user `d2ecf39af75b4ac29b9a65312fd3cf12`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:20.878736 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1a0ef631-0876-41fe-a1bc-46f4a9499eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:20.907020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388d262-72e6-453d-b49f-498c2a0cbee9 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:20.921788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:20.927178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a0a88-d37b-4a8a-b3ad-3d5f6edc37bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:20.928139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a0a88-d37b-4a8a-b3ad-3d5f6edc37bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:20.929120 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a0a88-d37b-4a8a-b3ad-3d5f6edc37bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:20.929453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a0a88-d37b-4a8a-b3ad-3d5f6edc37bf admin admin] PATH_INFO: `/v3/users/3513b60703944443ac2950859215eb2c` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:20.930889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a0a88-d37b-4a8a-b3ad-3d5f6edc37bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3513b60703944443ac2950859215eb2c)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:20.933134 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a0a88-d37b-4a8a-b3ad-3d5f6edc37bf admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:21.116231 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-795a0a88-d37b-4a8a-b3ad-3d5f6edc37bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3513b60703944443ac2950859215eb2c'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:21.170487 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 868/1753] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:20 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:21.171802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ef631-0876-41fe-a1bc-46f4a9499eca None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:21.173690 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1a0ef631-0876-41fe-a1bc-46f4a9499eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:21.185621 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-795a0a88-d37b-4a8a-b3ad-3d5f6edc37bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3513b60703944443ac2950859215eb2c'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:21.189797 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-795a0a88-d37b-4a8a-b3ad-3d5f6edc37bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3513b60703944443ac2950859215eb2c'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:21.216602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-795a0a88-d37b-4a8a-b3ad-3d5f6edc37bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3513b60703944443ac2950859215eb2c'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:21.226243 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-795a0a88-d37b-4a8a-b3ad-3d5f6edc37bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3513b60703944443ac2950859215eb2c'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:21.248781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 886/1754] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:20 2022] DELETE /identity/v3/users/3513b60703944443ac2950859215eb2c => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:21.249668 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a0a88-d37b-4a8a-b3ad-3d5f6edc37bf admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:21.258033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-795a0a88-d37b-4a8a-b3ad-3d5f6edc37bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:21.433408 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ef631-0876-41fe-a1bc-46f4a9499eca None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:21.482607 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:21.531407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a0a88-d37b-4a8a-b3ad-3d5f6edc37bf admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:21.548417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60460060-2602-4cd8-9a01-f470790de376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:21.549193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60460060-2602-4cd8-9a01-f470790de376 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:21.549476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60460060-2602-4cd8-9a01-f470790de376 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:21.549854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60460060-2602-4cd8-9a01-f470790de376 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:21.550608 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60460060-2602-4cd8-9a01-f470790de376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:21.552310 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60460060-2602-4cd8-9a01-f470790de376 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:21.600445 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 887/1755] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:21.601140 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60460060-2602-4cd8-9a01-f470790de376 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:21.615085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-60460060-2602-4cd8-9a01-f470790de376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:21.650867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60460060-2602-4cd8-9a01-f470790de376 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:21.684018 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea478d8-ca67-4a79-8c80-2a9b1f679247 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:21.685133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea478d8-ca67-4a79-8c80-2a9b1f679247 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:21.686799 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea478d8-ca67-4a79-8c80-2a9b1f679247 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:21.688233 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea478d8-ca67-4a79-8c80-2a9b1f679247 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:21.697086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-eea478d8-ca67-4a79-8c80-2a9b1f679247 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:21.701419 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b9218-5955-477f-a0ca-6b5621bff71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:21.717039 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b9218-5955-477f-a0ca-6b5621bff71e admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:21.718284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b9218-5955-477f-a0ca-6b5621bff71e admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:21.720391 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b9218-5955-477f-a0ca-6b5621bff71e admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:21.726893 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b9218-5955-477f-a0ca-6b5621bff71e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:21.739396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b9218-5955-477f-a0ca-6b5621bff71e admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:21.824638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 888/1756] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:21 2022] POST /identity/v3/projects => generated 403 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:21.832619 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79e287-0cc8-4202-925e-9aac212155bf None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:21.833277 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79e287-0cc8-4202-925e-9aac212155bf None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:21.837334 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79e287-0cc8-4202-925e-9aac212155bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:21.883559 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-8a79e287-0cc8-4202-925e-9aac212155bf None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:21.890736 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8a79e287-0cc8-4202-925e-9aac212155bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:21.909618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea478d8-ca67-4a79-8c80-2a9b1f679247 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:21.917020 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea478d8-ca67-4a79-8c80-2a9b1f679247 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:21.965208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 869/1757] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 794 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:21.966364 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea478d8-ca67-4a79-8c80-2a9b1f679247 service cinder] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:22.016257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea478d8-ca67-4a79-8c80-2a9b1f679247 service cinder] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:22.146050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:22.169718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 889/1758] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:22.172902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79e287-0cc8-4202-925e-9aac212155bf None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:22.210677 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79e287-0cc8-4202-925e-9aac212155bf None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:22.226522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65650e78-8590-4faf-82eb-e17ad5a5334c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:22.227610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65650e78-8590-4faf-82eb-e17ad5a5334c admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:22.228261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65650e78-8590-4faf-82eb-e17ad5a5334c admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:22.228781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65650e78-8590-4faf-82eb-e17ad5a5334c admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:22.229737 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65650e78-8590-4faf-82eb-e17ad5a5334c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:22.231949 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65650e78-8590-4faf-82eb-e17ad5a5334c admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:22.283977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1e38a-197d-426c-a266-7dce80f6cc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:22.284907 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1e38a-197d-426c-a266-7dce80f6cc21 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:22.285305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1e38a-197d-426c-a266-7dce80f6cc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:22.285589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1e38a-197d-426c-a266-7dce80f6cc21 admin admin] PATH_INFO: `/v3/projects/22a22ab062a94af5a161780ca7589cb2/users/c3342cd47c3a4650882d7b9df6d7e087/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:22.292847 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 890/1759] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:22 2022] POST /identity/v3/users => generated 308 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:22.295251 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65650e78-8590-4faf-82eb-e17ad5a5334c admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:22.300654 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-65650e78-8590-4faf-82eb-e17ad5a5334c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:22.332507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1e38a-197d-426c-a266-7dce80f6cc21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22a22ab062a94af5a161780ca7589cb2, user_id=c3342cd47c3a4650882d7b9df6d7e087, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:22.344588 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1e38a-197d-426c-a266-7dce80f6cc21 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:22.364736 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 870/1760] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:21 2022] PUT /identity/v3/projects/22a22ab062a94af5a161780ca7589cb2/users/c3342cd47c3a4650882d7b9df6d7e087/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:22.367014 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1e38a-197d-426c-a266-7dce80f6cc21 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:22.514838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65650e78-8590-4faf-82eb-e17ad5a5334c admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:22.607406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695d04a-8838-42c5-a468-edfc6568759c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:22.608274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695d04a-8838-42c5-a468-edfc6568759c service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:22.608572 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695d04a-8838-42c5-a468-edfc6568759c service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:22.608854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695d04a-8838-42c5-a468-edfc6568759c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:22.625090 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695d04a-8838-42c5-a468-edfc6568759c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:22.628460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695d04a-8838-42c5-a468-edfc6568759c service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:22.637496 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1e38a-197d-426c-a266-7dce80f6cc21 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:22.642101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 891/1761] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:22.643570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695d04a-8838-42c5-a468-edfc6568759c service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:22.649672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:22.661395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695d04a-8838-42c5-a468-edfc6568759c service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:22.667836 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04fb11a-18a4-4bad-a09a-7c2ba9eec7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:22.670578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fb11a-18a4-4bad-a09a-7c2ba9eec7ee admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:22.676649 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fb11a-18a4-4bad-a09a-7c2ba9eec7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:22.677151 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fb11a-18a4-4bad-a09a-7c2ba9eec7ee admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:22.680134 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04fb11a-18a4-4bad-a09a-7c2ba9eec7ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:22.683075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04fb11a-18a4-4bad-a09a-7c2ba9eec7ee admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:22.694973 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc3de7-0adf-466c-9b84-8500c9bab3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:22.697416 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc3de7-0adf-466c-9b84-8500c9bab3bd admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:22.700171 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc3de7-0adf-466c-9b84-8500c9bab3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:22.701109 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc3de7-0adf-466c-9b84-8500c9bab3bd admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:22.704050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc3de7-0adf-466c-9b84-8500c9bab3bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:22.707155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc3de7-0adf-466c-9b84-8500c9bab3bd admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:22.727400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 892/1762] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:22.728627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc3de7-0adf-466c-9b84-8500c9bab3bd admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:22.730697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 871/1763] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:22.733096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04fb11a-18a4-4bad-a09a-7c2ba9eec7ee admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:22.765948 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc3de7-0adf-466c-9b84-8500c9bab3bd admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:22.766652 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04fb11a-18a4-4bad-a09a-7c2ba9eec7ee admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:22.775939 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a5a0b-a87c-4169-8d42-92e79baef8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:22.777334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a5a0b-a87c-4169-8d42-92e79baef8ba admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:22.777772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a5a0b-a87c-4169-8d42-92e79baef8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:22.779022 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a5a0b-a87c-4169-8d42-92e79baef8ba admin admin] PATH_INFO: `/v3/projects/2e63eb9f6ae448ee82a2e6a24001bfc9/users/acd9d9dd27ee492ea23f889e3a8114f8/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:22.784902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dc442-67e4-40cf-b0cd-96a7223112b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:22.790713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dc442-67e4-40cf-b0cd-96a7223112b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:22.791016 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dc442-67e4-40cf-b0cd-96a7223112b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:22.792470 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dc442-67e4-40cf-b0cd-96a7223112b6 admin admin] PATH_INFO: `/v3/projects/22a22ab062a94af5a161780ca7589cb2/users/c3342cd47c3a4650882d7b9df6d7e087/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:22.830008 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6a5a0b-a87c-4169-8d42-92e79baef8ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e63eb9f6ae448ee82a2e6a24001bfc9, user_id=acd9d9dd27ee492ea23f889e3a8114f8, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:22.837998 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6a5a0b-a87c-4169-8d42-92e79baef8ba admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:22.838217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0dc442-67e4-40cf-b0cd-96a7223112b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22a22ab062a94af5a161780ca7589cb2, user_id=c3342cd47c3a4650882d7b9df6d7e087, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:22.841464 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0dc442-67e4-40cf-b0cd-96a7223112b6 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:22.872071 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 872/1764] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:22 2022] PUT /identity/v3/projects/2e63eb9f6ae448ee82a2e6a24001bfc9/users/acd9d9dd27ee492ea23f889e3a8114f8/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:22.877497 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a5a0b-a87c-4169-8d42-92e79baef8ba admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:22.910966 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 893/1765] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:22 2022] PUT /identity/v3/projects/22a22ab062a94af5a161780ca7589cb2/users/c3342cd47c3a4650882d7b9df6d7e087/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:22.927062 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fbcc8-b912-4c8c-aea5-1b716bcd15ca None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:22.941547 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fbcc8-b912-4c8c-aea5-1b716bcd15ca None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:22.941949 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fbcc8-b912-4c8c-aea5-1b716bcd15ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:23.067139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-251fbcc8-b912-4c8c-aea5-1b716bcd15ca None None] MFA Rules not processed for user `c3342cd47c3a4650882d7b9df6d7e087`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:23.069226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-251fbcc8-b912-4c8c-aea5-1b716bcd15ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:23.216501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a5a0b-a87c-4169-8d42-92e79baef8ba admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:23.307005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:23.376902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 894/1766] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:22 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:23.386028 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fbcc8-b912-4c8c-aea5-1b716bcd15ca None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:23.388070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-251fbcc8-b912-4c8c-aea5-1b716bcd15ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:23.415394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85526949-9592-44a6-82ab-8119b79ca0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:23.416286 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85526949-9592-44a6-82ab-8119b79ca0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:23.416697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85526949-9592-44a6-82ab-8119b79ca0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:23.417077 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85526949-9592-44a6-82ab-8119b79ca0d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:23.418018 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85526949-9592-44a6-82ab-8119b79ca0d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:23.423627 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85526949-9592-44a6-82ab-8119b79ca0d0 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:23.473521 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 873/1767] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 596 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:48:23.478117 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85526949-9592-44a6-82ab-8119b79ca0d0 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:23.510986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85526949-9592-44a6-82ab-8119b79ca0d0 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:23.531884 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a1b10-5a44-4668-9007-a39afdb68402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:23.533019 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a1b10-5a44-4668-9007-a39afdb68402 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:23.538173 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a1b10-5a44-4668-9007-a39afdb68402 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:23.539096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a1b10-5a44-4668-9007-a39afdb68402 admin admin] PATH_INFO: `/v3/projects/2e63eb9f6ae448ee82a2e6a24001bfc9/users/acd9d9dd27ee492ea23f889e3a8114f8/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:23.545252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a1b10-5a44-4668-9007-a39afdb68402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e63eb9f6ae448ee82a2e6a24001bfc9, user_id=acd9d9dd27ee492ea23f889e3a8114f8, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:23.571108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fbcc8-b912-4c8c-aea5-1b716bcd15ca None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:23.575853 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75322bb-efc3-4bab-adb4-aaa0c4695199 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:23.576646 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75322bb-efc3-4bab-adb4-aaa0c4695199 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:23.576988 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75322bb-efc3-4bab-adb4-aaa0c4695199 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:23.577326 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75322bb-efc3-4bab-adb4-aaa0c4695199 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:23.579077 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a1b10-5a44-4668-9007-a39afdb68402 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:23.592626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75322bb-efc3-4bab-adb4-aaa0c4695199 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:23.594710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75322bb-efc3-4bab-adb4-aaa0c4695199 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:23.614605 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 895/1768] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:23 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:48:23.633835 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 874/1769] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:23 2022] PUT /identity/v3/projects/2e63eb9f6ae448ee82a2e6a24001bfc9/users/acd9d9dd27ee492ea23f889e3a8114f8/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:48:24.436098 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75322bb-efc3-4bab-adb4-aaa0c4695199 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:24.438165 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e75322bb-efc3-4bab-adb4-aaa0c4695199 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:24.477720 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75322bb-efc3-4bab-adb4-aaa0c4695199 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:24.712159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbff752-4d3c-43f5-ba07-0ba676b39edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:24.713455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbff752-4d3c-43f5-ba07-0ba676b39edd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:24.714160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbff752-4d3c-43f5-ba07-0ba676b39edd admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:24.714644 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbff752-4d3c-43f5-ba07-0ba676b39edd admin admin] PATH_INFO: `/v3/projects/6481f389f5524085b205c153f4ff8556` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:24.716279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbff752-4d3c-43f5-ba07-0ba676b39edd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6481f389f5524085b205c153f4ff8556)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:24.718974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbff752-4d3c-43f5-ba07-0ba676b39edd admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:24.735511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:24.860977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-fcbff752-4d3c-43f5-ba07-0ba676b39edd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6481f389f5524085b205c153f4ff8556', 'request_id': 'req-fcbff752-4d3c-43f5-ba07-0ba676b39edd'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:24.893598 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-fcbff752-4d3c-43f5-ba07-0ba676b39edd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6481f389f5524085b205c153f4ff8556', 'request_id': 'req-fcbff752-4d3c-43f5-ba07-0ba676b39edd'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:24.902598 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-fcbff752-4d3c-43f5-ba07-0ba676b39edd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6481f389f5524085b205c153f4ff8556', 'request_id': 'req-fcbff752-4d3c-43f5-ba07-0ba676b39edd'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:24.961734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a1b10-5a44-4668-9007-a39afdb68402 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:24.969951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-fcbff752-4d3c-43f5-ba07-0ba676b39edd admin admin] The token cache is being invalidate because project 6481f389f5524085b205c153f4ff8556 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:24.973547 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-fcbff752-4d3c-43f5-ba07-0ba676b39edd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:24.977333 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-fcbff752-4d3c-43f5-ba07-0ba676b39edd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:24.977609 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a1b10-5a44-4668-9007-a39afdb68402 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:24.986554 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 896/1770] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:24 2022] DELETE /identity/v3/projects/6481f389f5524085b205c153f4ff8556 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:24.991840 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbff752-4d3c-43f5-ba07-0ba676b39edd admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:24.997491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-fcbff752-4d3c-43f5-ba07-0ba676b39edd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:25.165022 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f08d8-b50a-4814-a2e9-d398ccb52074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:25.166380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f08d8-b50a-4814-a2e9-d398ccb52074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:25.167125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f08d8-b50a-4814-a2e9-d398ccb52074 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:25.167650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f08d8-b50a-4814-a2e9-d398ccb52074 admin admin] PATH_INFO: `/v3/users/898af1d0e74f415c8555d5e4a35e0232` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:25.172834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f08d8-b50a-4814-a2e9-d398ccb52074 admin admin] RBAC: Authorizing `identity:delete_user(user_id=898af1d0e74f415c8555d5e4a35e0232)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:25.175150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f08d8-b50a-4814-a2e9-d398ccb52074 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:25.314628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-cc1f08d8-b50a-4814-a2e9-d398ccb52074 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '898af1d0e74f415c8555d5e4a35e0232'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:25.352051 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-cc1f08d8-b50a-4814-a2e9-d398ccb52074 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '898af1d0e74f415c8555d5e4a35e0232'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:25.369416 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbff752-4d3c-43f5-ba07-0ba676b39edd admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:25.378355 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-cc1f08d8-b50a-4814-a2e9-d398ccb52074 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '898af1d0e74f415c8555d5e4a35e0232'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:25.388358 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-cc1f08d8-b50a-4814-a2e9-d398ccb52074 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '898af1d0e74f415c8555d5e4a35e0232'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:25.393682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-cc1f08d8-b50a-4814-a2e9-d398ccb52074 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '898af1d0e74f415c8555d5e4a35e0232'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:25.403693 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fad2af-8264-402a-81ed-d004ca8d82ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:25.404620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fad2af-8264-402a-81ed-d004ca8d82ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:25.414215 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fad2af-8264-402a-81ed-d004ca8d82ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:25.414544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fad2af-8264-402a-81ed-d004ca8d82ae admin admin] PATH_INFO: `/v3/users/c97a085ef91e4c8392091477e6596c57` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:25.426845 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fad2af-8264-402a-81ed-d004ca8d82ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=c97a085ef91e4c8392091477e6596c57)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:25.431885 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fad2af-8264-402a-81ed-d004ca8d82ae admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:25.450515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 875/1771] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:24 2022] DELETE /identity/v3/users/898af1d0e74f415c8555d5e4a35e0232 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:25.455222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03358454-f22b-4e51-a0e6-a5d650183f92 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:25.455519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03358454-f22b-4e51-a0e6-a5d650183f92 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:25.455854 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03358454-f22b-4e51-a0e6-a5d650183f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:25.485602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-03358454-f22b-4e51-a0e6-a5d650183f92 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:25.487694 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-03358454-f22b-4e51-a0e6-a5d650183f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:25.534477 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:25.591412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-98fad2af-8264-402a-81ed-d004ca8d82ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c97a085ef91e4c8392091477e6596c57'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:25.624163 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-98fad2af-8264-402a-81ed-d004ca8d82ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c97a085ef91e4c8392091477e6596c57'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:25.625460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-98fad2af-8264-402a-81ed-d004ca8d82ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c97a085ef91e4c8392091477e6596c57'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:25.628230 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 876/1772] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:48:25.641694 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03358454-f22b-4e51-a0e6-a5d650183f92 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:25.642789 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-98fad2af-8264-402a-81ed-d004ca8d82ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c97a085ef91e4c8392091477e6596c57'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:25.643700 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-03358454-f22b-4e51-a0e6-a5d650183f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:25.653820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-98fad2af-8264-402a-81ed-d004ca8d82ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c97a085ef91e4c8392091477e6596c57'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:25.663187 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 897/1773] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:24 2022] DELETE /identity/v3/users/c97a085ef91e4c8392091477e6596c57 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:25.739217 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03358454-f22b-4e51-a0e6-a5d650183f92 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:25.814382 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7c079-5ed3-4de8-b125-e5675feeb182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:25.815229 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7c079-5ed3-4de8-b125-e5675feeb182 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:25.816549 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7c079-5ed3-4de8-b125-e5675feeb182 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:25.816549 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7c079-5ed3-4de8-b125-e5675feeb182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:25.818762 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-6bc7c079-5ed3-4de8-b125-e5675feeb182 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:25.950873 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7c079-5ed3-4de8-b125-e5675feeb182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:25.952822 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7c079-5ed3-4de8-b125-e5675feeb182 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:25.966768 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 877/1774] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:26.295927 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fad2af-8264-402a-81ed-d004ca8d82ae admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:26.298393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-98fad2af-8264-402a-81ed-d004ca8d82ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:26.336851 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fad2af-8264-402a-81ed-d004ca8d82ae admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:26.349907 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4d1ce-9868-4e97-8098-e661c5e6e7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:26.351789 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4d1ce-9868-4e97-8098-e661c5e6e7f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:26.352248 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4d1ce-9868-4e97-8098-e661c5e6e7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:26.352670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4d1ce-9868-4e97-8098-e661c5e6e7f5 admin admin] PATH_INFO: `/v3/projects/cd25c5eab42c4d319217c324cb13acb8` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:26.354216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4d1ce-9868-4e97-8098-e661c5e6e7f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd25c5eab42c4d319217c324cb13acb8)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:26.356368 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4d1ce-9868-4e97-8098-e661c5e6e7f5 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:26.383826 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:26.415762 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f84b7-eddf-410a-be85-674216ba2250 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:26.416119 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f84b7-eddf-410a-be85-674216ba2250 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:26.416952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f84b7-eddf-410a-be85-674216ba2250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:26.451582 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-84e4d1ce-9868-4e97-8098-e661c5e6e7f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd25c5eab42c4d319217c324cb13acb8', 'request_id': 'req-84e4d1ce-9868-4e97-8098-e661c5e6e7f5'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:26.458228 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-5e2f84b7-eddf-410a-be85-674216ba2250 None None] MFA Rules not processed for user `acd9d9dd27ee492ea23f889e3a8114f8`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:26.459737 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-5e2f84b7-eddf-410a-be85-674216ba2250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:26.461754 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-84e4d1ce-9868-4e97-8098-e661c5e6e7f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd25c5eab42c4d319217c324cb13acb8', 'request_id': 'req-84e4d1ce-9868-4e97-8098-e661c5e6e7f5'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:26.472870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-84e4d1ce-9868-4e97-8098-e661c5e6e7f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd25c5eab42c4d319217c324cb13acb8', 'request_id': 'req-84e4d1ce-9868-4e97-8098-e661c5e6e7f5'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:26.490469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-84e4d1ce-9868-4e97-8098-e661c5e6e7f5 admin admin] The token cache is being invalidate because project cd25c5eab42c4d319217c324cb13acb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:26.492062 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-84e4d1ce-9868-4e97-8098-e661c5e6e7f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:26.493021 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-84e4d1ce-9868-4e97-8098-e661c5e6e7f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:26.495298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:26.496863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 898/1775] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:26 2022] DELETE /identity/v3/projects/cd25c5eab42c4d319217c324cb13acb8 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:26.502011 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff7989-d23c-4529-b261-8d5e26a49f8f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:26.502857 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff7989-d23c-4529-b261-8d5e26a49f8f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:26.503942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff7989-d23c-4529-b261-8d5e26a49f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:26.524876 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-00ff7989-d23c-4529-b261-8d5e26a49f8f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:26.526616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-00ff7989-d23c-4529-b261-8d5e26a49f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:26.566480 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 878/1776] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:26 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:26.626225 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f84b7-eddf-410a-be85-674216ba2250 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:26.627874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-5e2f84b7-eddf-410a-be85-674216ba2250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:26.674478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 899/1777] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:26.676559 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ffa86-bd65-4ae7-8701-82f4ba2ca59a None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:26.681736 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ffa86-bd65-4ae7-8701-82f4ba2ca59a None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:26.682573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ffa86-bd65-4ae7-8701-82f4ba2ca59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:26.705489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-ff6ffa86-bd65-4ae7-8701-82f4ba2ca59a None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:26.706762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-ff6ffa86-bd65-4ae7-8701-82f4ba2ca59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:26.716299 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 900/1778] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:26.720247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ffa86-bd65-4ae7-8701-82f4ba2ca59a None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:26.730918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ffa86-bd65-4ae7-8701-82f4ba2ca59a None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:26.736600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c8bc8-8c15-43cb-b540-2b7eed4aaf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:26.737279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c8bc8-8c15-43cb-b540-2b7eed4aaf8b admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:26.737575 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c8bc8-8c15-43cb-b540-2b7eed4aaf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:26.737884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c8bc8-8c15-43cb-b540-2b7eed4aaf8b admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:26.738685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c8bc8-8c15-43cb-b540-2b7eed4aaf8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:48:26.738996 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c8bc8-8c15-43cb-b540-2b7eed4aaf8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:26.749888 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c8bc8-8c15-43cb-b540-2b7eed4aaf8b admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:26.751307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f84b7-eddf-410a-be85-674216ba2250 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:26.756741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ecbfee-c1a7-42dd-8478-c34902d0d911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:26.757936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecbfee-c1a7-42dd-8478-c34902d0d911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:26.758222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecbfee-c1a7-42dd-8478-c34902d0d911 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:26.758508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecbfee-c1a7-42dd-8478-c34902d0d911 admin admin] PATH_INFO: `/v3/projects/f9983cf7d78d470887d346186b71b938` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:26.758908 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 901/1779] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:48:26.759622 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c8bc8-8c15-43cb-b540-2b7eed4aaf8b admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:26.759782 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ecbfee-c1a7-42dd-8478-c34902d0d911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9983cf7d78d470887d346186b71b938)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:26.761690 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ecbfee-c1a7-42dd-8478-c34902d0d911 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:26.770604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c8bc8-8c15-43cb-b540-2b7eed4aaf8b admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:26.776614 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706db07-2c2c-43f7-b03d-08443f5025c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:26.777566 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706db07-2c2c-43f7-b03d-08443f5025c6 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:26.777566 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706db07-2c2c-43f7-b03d-08443f5025c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:26.777851 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706db07-2c2c-43f7-b03d-08443f5025c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:26.778541 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3706db07-2c2c-43f7-b03d-08443f5025c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:26.780443 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3706db07-2c2c-43f7-b03d-08443f5025c6 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:26.812317 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 902/1780] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:26 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:26.816784 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706db07-2c2c-43f7-b03d-08443f5025c6 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:26.828991 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-98ecbfee-c1a7-42dd-8478-c34902d0d911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9983cf7d78d470887d346186b71b938', 'request_id': 'req-98ecbfee-c1a7-42dd-8478-c34902d0d911'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:26.836834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-98ecbfee-c1a7-42dd-8478-c34902d0d911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9983cf7d78d470887d346186b71b938', 'request_id': 'req-98ecbfee-c1a7-42dd-8478-c34902d0d911'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:26.843396 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-98ecbfee-c1a7-42dd-8478-c34902d0d911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9983cf7d78d470887d346186b71b938', 'request_id': 'req-98ecbfee-c1a7-42dd-8478-c34902d0d911'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:26.861387 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-98ecbfee-c1a7-42dd-8478-c34902d0d911 admin admin] The token cache is being invalidate because project f9983cf7d78d470887d346186b71b938 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:26.861718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-98ecbfee-c1a7-42dd-8478-c34902d0d911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:26.862420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-98ecbfee-c1a7-42dd-8478-c34902d0d911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:26.868907 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 879/1781] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:26 2022] DELETE /identity/v3/projects/f9983cf7d78d470887d346186b71b938 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:26.870125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ecbfee-c1a7-42dd-8478-c34902d0d911 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:26.871600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-98ecbfee-c1a7-42dd-8478-c34902d0d911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:26.894063 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:26.898203 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706db07-2c2c-43f7-b03d-08443f5025c6 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:26.904232 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3026bc-db6f-4969-acad-b1db55afca5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:26.905157 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3026bc-db6f-4969-acad-b1db55afca5c admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:26.905459 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3026bc-db6f-4969-acad-b1db55afca5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:26.906204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3026bc-db6f-4969-acad-b1db55afca5c admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:26.907101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3026bc-db6f-4969-acad-b1db55afca5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:26.909604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3026bc-db6f-4969-acad-b1db55afca5c admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:26.935247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 903/1782] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:26 2022] POST /identity/v3/users => generated 294 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:48:26.939019 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:26.939279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3026bc-db6f-4969-acad-b1db55afca5c admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:26.941333 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-bb3026bc-db6f-4969-acad-b1db55afca5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:26.954925 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3026bc-db6f-4969-acad-b1db55afca5c admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:26.955431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ecbfee-c1a7-42dd-8478-c34902d0d911 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:26.961617 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987514aa-7980-4afb-910b-3ef1bc1a021f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:26.961827 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234f567-22ed-4dab-b6f6-bc56ad473d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:26.962267 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987514aa-7980-4afb-910b-3ef1bc1a021f admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:26.962550 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987514aa-7980-4afb-910b-3ef1bc1a021f admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:26.962727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234f567-22ed-4dab-b6f6-bc56ad473d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:26.962838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987514aa-7980-4afb-910b-3ef1bc1a021f admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:26.963200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234f567-22ed-4dab-b6f6-bc56ad473d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:26.963488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234f567-22ed-4dab-b6f6-bc56ad473d19 admin admin] PATH_INFO: `/v3/users/c3342cd47c3a4650882d7b9df6d7e087` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:26.963580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987514aa-7980-4afb-910b-3ef1bc1a021f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:26.964763 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234f567-22ed-4dab-b6f6-bc56ad473d19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3342cd47c3a4650882d7b9df6d7e087)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:26.965121 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987514aa-7980-4afb-910b-3ef1bc1a021f admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:26.966923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234f567-22ed-4dab-b6f6-bc56ad473d19 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:26.977458 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 904/1783] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:48:26.981384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987514aa-7980-4afb-910b-3ef1bc1a021f admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:26.993404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987514aa-7980-4afb-910b-3ef1bc1a021f admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:26.999407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544b21f-be3d-4da9-b25b-1acd4fe9c985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:27.000272 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544b21f-be3d-4da9-b25b-1acd4fe9c985 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:27.001165 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544b21f-be3d-4da9-b25b-1acd4fe9c985 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:27.001478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544b21f-be3d-4da9-b25b-1acd4fe9c985 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:27.002470 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544b21f-be3d-4da9-b25b-1acd4fe9c985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:27.004257 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544b21f-be3d-4da9-b25b-1acd4fe9c985 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:27.017180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 905/1784] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:48:27.018274 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-9234f567-22ed-4dab-b6f6-bc56ad473d19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3342cd47c3a4650882d7b9df6d7e087'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:27.021449 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544b21f-be3d-4da9-b25b-1acd4fe9c985 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:27.031599 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-9234f567-22ed-4dab-b6f6-bc56ad473d19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3342cd47c3a4650882d7b9df6d7e087'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:27.032466 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544b21f-be3d-4da9-b25b-1acd4fe9c985 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:27.041218 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-9234f567-22ed-4dab-b6f6-bc56ad473d19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3342cd47c3a4650882d7b9df6d7e087'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:27.041400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4036d04-b453-4093-b6ae-8e24004c76fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:27.042251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-9234f567-22ed-4dab-b6f6-bc56ad473d19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3342cd47c3a4650882d7b9df6d7e087'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:27.042405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4036d04-b453-4093-b6ae-8e24004c76fc admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:27.042616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4036d04-b453-4093-b6ae-8e24004c76fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:27.042888 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4036d04-b453-4093-b6ae-8e24004c76fc admin admin] PATH_INFO: `/v3/projects/f6ebada18659446ca3d1ca54af3cd71a/users/09f525f71e1b43a2ac32c2682c323b85/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:27.042986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-9234f567-22ed-4dab-b6f6-bc56ad473d19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3342cd47c3a4650882d7b9df6d7e087'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:27.059399 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 880/1785] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:26 2022] DELETE /identity/v3/users/c3342cd47c3a4650882d7b9df6d7e087 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:27.059750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4036d04-b453-4093-b6ae-8e24004c76fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ebada18659446ca3d1ca54af3cd71a, user_id=09f525f71e1b43a2ac32c2682c323b85, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:27.062842 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4036d04-b453-4093-b6ae-8e24004c76fc admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:27.080335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 906/1786] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:27 2022] PUT /identity/v3/projects/f6ebada18659446ca3d1ca54af3cd71a/users/09f525f71e1b43a2ac32c2682c323b85/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:27.093050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1754067-d72e-4ecf-8c0a-db62889e59b6 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:27.093617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1754067-d72e-4ecf-8c0a-db62889e59b6 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:27.094099 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1754067-d72e-4ecf-8c0a-db62889e59b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:27.138353 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-d1754067-d72e-4ecf-8c0a-db62889e59b6 None None] MFA Rules not processed for user `09f525f71e1b43a2ac32c2682c323b85`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:27.139442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d1754067-d72e-4ecf-8c0a-db62889e59b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:27.217621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 881/1787] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:27.218720 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4036d04-b453-4093-b6ae-8e24004c76fc admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:27.220566 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-b4036d04-b453-4093-b6ae-8e24004c76fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:27.310756 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4036d04-b453-4093-b6ae-8e24004c76fc admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:27.315874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff871afc-1c7b-4b3e-8548-fb1a0dd0b3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:27.316765 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff871afc-1c7b-4b3e-8548-fb1a0dd0b3a4 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:27.316970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff871afc-1c7b-4b3e-8548-fb1a0dd0b3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:27.317285 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff871afc-1c7b-4b3e-8548-fb1a0dd0b3a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:27.319509 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff871afc-1c7b-4b3e-8548-fb1a0dd0b3a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:27.321715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff871afc-1c7b-4b3e-8548-fb1a0dd0b3a4 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:27.357992 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 907/1788] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:27 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:48:27.362635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1754067-d72e-4ecf-8c0a-db62889e59b6 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:27.375491 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1754067-d72e-4ecf-8c0a-db62889e59b6 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:27.383995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a6306-cc9b-405a-9efd-9e1bedb7f14a None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:27.384543 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a6306-cc9b-405a-9efd-9e1bedb7f14a None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:27.385040 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a6306-cc9b-405a-9efd-9e1bedb7f14a None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:27.387321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 908/1789] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jun 27 15:48:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:48:27.397776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88601b5c-ea89-4d36-9801-5f0b3da4a1a7 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:27.398138 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88601b5c-ea89-4d36-9801-5f0b3da4a1a7 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:27.398479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88601b5c-ea89-4d36-9801-5f0b3da4a1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:27.431791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-88601b5c-ea89-4d36-9801-5f0b3da4a1a7 None None] MFA Rules not processed for user `ffafb68f1bb941588a5217ad83ad907c`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:27.436217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-88601b5c-ea89-4d36-9801-5f0b3da4a1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:27.467256 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe435c2-6acd-4772-8178-7853da0e13d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:27.468173 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe435c2-6acd-4772-8178-7853da0e13d5 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:27.468512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe435c2-6acd-4772-8178-7853da0e13d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:27.468909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe435c2-6acd-4772-8178-7853da0e13d5 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:27.469740 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe435c2-6acd-4772-8178-7853da0e13d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:27.471792 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe435c2-6acd-4772-8178-7853da0e13d5 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:27.482198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:27.501090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 882/1790] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:27 2022] POST /identity/v3/users => generated 299 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:27.501090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe435c2-6acd-4772-8178-7853da0e13d5 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:27.502835 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1fe435c2-6acd-4772-8178-7853da0e13d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:27.516231 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe435c2-6acd-4772-8178-7853da0e13d5 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:27.522529 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12c47a-ff36-45ca-aa05-5e7b7ee75d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:27.523647 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12c47a-ff36-45ca-aa05-5e7b7ee75d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:27.524013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12c47a-ff36-45ca-aa05-5e7b7ee75d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:27.524377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12c47a-ff36-45ca-aa05-5e7b7ee75d7a admin admin] PATH_INFO: `/v3/projects/22a22ab062a94af5a161780ca7589cb2` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:27.525810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12c47a-ff36-45ca-aa05-5e7b7ee75d7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=22a22ab062a94af5a161780ca7589cb2)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:27.527727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12c47a-ff36-45ca-aa05-5e7b7ee75d7a admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:27.542578 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 909/1791] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jun 27 15:48:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:27.543350 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88601b5c-ea89-4d36-9801-5f0b3da4a1a7 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:27.558031 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88601b5c-ea89-4d36-9801-5f0b3da4a1a7 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:27.565083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ea99f-473c-4d91-bacd-7bf4c465d299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:27.565828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ea99f-473c-4d91-bacd-7bf4c465d299 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:27.566170 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ea99f-473c-4d91-bacd-7bf4c465d299 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:27.566452 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ea99f-473c-4d91-bacd-7bf4c465d299 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:27.567194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ea99f-473c-4d91-bacd-7bf4c465d299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:27.568883 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ea99f-473c-4d91-bacd-7bf4c465d299 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:27.580172 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 910/1792] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:27.580844 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ea99f-473c-4d91-bacd-7bf4c465d299 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:27.582615 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-a11ea99f-473c-4d91-bacd-7bf4c465d299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:27.591226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-bc12c47a-ff36-45ca-aa05-5e7b7ee75d7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22a22ab062a94af5a161780ca7589cb2', 'request_id': 'req-bc12c47a-ff36-45ca-aa05-5e7b7ee75d7a'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:27.598406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ea99f-473c-4d91-bacd-7bf4c465d299 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:27.598696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-bc12c47a-ff36-45ca-aa05-5e7b7ee75d7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22a22ab062a94af5a161780ca7589cb2', 'request_id': 'req-bc12c47a-ff36-45ca-aa05-5e7b7ee75d7a'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:27.605734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-bc12c47a-ff36-45ca-aa05-5e7b7ee75d7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22a22ab062a94af5a161780ca7589cb2', 'request_id': 'req-bc12c47a-ff36-45ca-aa05-5e7b7ee75d7a'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:27.608794 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:27.634354 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-bc12c47a-ff36-45ca-aa05-5e7b7ee75d7a admin admin] The token cache is being invalidate because project 22a22ab062a94af5a161780ca7589cb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:27.634775 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-bc12c47a-ff36-45ca-aa05-5e7b7ee75d7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:27.635534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-bc12c47a-ff36-45ca-aa05-5e7b7ee75d7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:27.638373 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 883/1793] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:27 2022] DELETE /identity/v3/projects/22a22ab062a94af5a161780ca7589cb2 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:27.639358 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12c47a-ff36-45ca-aa05-5e7b7ee75d7a admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:27.640977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-bc12c47a-ff36-45ca-aa05-5e7b7ee75d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:27.683405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc45de-c7a9-43f8-bdab-cca8016f60c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffafb68f1bb941588a5217ad83ad907c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:27.684220 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc45de-c7a9-43f8-bdab-cca8016f60c2 service placement] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:27.684575 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc45de-c7a9-43f8-bdab-cca8016f60c2 service placement] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:27.684925 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc45de-c7a9-43f8-bdab-cca8016f60c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:27.687053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-65bc45de-c7a9-43f8-bdab-cca8016f60c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:27.699595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc45de-c7a9-43f8-bdab-cca8016f60c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:27.701319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc45de-c7a9-43f8-bdab-cca8016f60c2 service placement] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:27.716792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 911/1794] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:27.718685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f7238-246a-4aab-a74a-88e80f347b6d None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:27.718917 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f7238-246a-4aab-a74a-88e80f347b6d None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:27.719190 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f7238-246a-4aab-a74a-88e80f347b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:27.738664 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12c47a-ff36-45ca-aa05-5e7b7ee75d7a admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:27.739586 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-618f7238-246a-4aab-a74a-88e80f347b6d None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:27.740515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-618f7238-246a-4aab-a74a-88e80f347b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:27.743810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b5515-a133-4bcf-8b91-77b8015ab265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:27.744871 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b5515-a133-4bcf-8b91-77b8015ab265 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:27.745162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b5515-a133-4bcf-8b91-77b8015ab265 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:27.745439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b5515-a133-4bcf-8b91-77b8015ab265 admin admin] PATH_INFO: `/v3/projects/76d864ec1fc448e2a386ae97334db442/users/d18e01e679fe4032bd82934f2df8e606/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:27.770930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b5515-a133-4bcf-8b91-77b8015ab265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d864ec1fc448e2a386ae97334db442, user_id=d18e01e679fe4032bd82934f2df8e606, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:27.773874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b5515-a133-4bcf-8b91-77b8015ab265 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:27.779015 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 912/1795] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:27.786888 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393810b-706c-4bac-bc58-9e878888798a None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:27.787182 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393810b-706c-4bac-bc58-9e878888798a None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:27.787460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393810b-706c-4bac-bc58-9e878888798a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:27.810612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 884/1796] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:27 2022] PUT /identity/v3/projects/76d864ec1fc448e2a386ae97334db442/users/d18e01e679fe4032bd82934f2df8e606/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:27.812173 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b5515-a133-4bcf-8b91-77b8015ab265 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:27.830395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-9393810b-706c-4bac-bc58-9e878888798a None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:27.831495 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-9393810b-706c-4bac-bc58-9e878888798a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:27.896357 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:27.903806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b5515-a133-4bcf-8b91-77b8015ab265 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:27.920229 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545a77d-3659-42b2-9ed6-f93727278257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:27.921012 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545a77d-3659-42b2-9ed6-f93727278257 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:27.921481 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545a77d-3659-42b2-9ed6-f93727278257 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:27.921805 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545a77d-3659-42b2-9ed6-f93727278257 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:27.922611 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545a77d-3659-42b2-9ed6-f93727278257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:27.924320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545a77d-3659-42b2-9ed6-f93727278257 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:27.938982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 885/1797] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:27.945544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545a77d-3659-42b2-9ed6-f93727278257 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:27.965271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545a77d-3659-42b2-9ed6-f93727278257 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:27.970340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 913/1798] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:27.974400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393810b-706c-4bac-bc58-9e878888798a None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:27.976658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae8e08-c551-4de0-8c8d-ce2ac0740303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:27.978683 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae8e08-c551-4de0-8c8d-ce2ac0740303 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:27.978946 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae8e08-c551-4de0-8c8d-ce2ac0740303 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:27.979284 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae8e08-c551-4de0-8c8d-ce2ac0740303 admin admin] PATH_INFO: `/v3/projects/76d864ec1fc448e2a386ae97334db442/users/d18e01e679fe4032bd82934f2df8e606/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:27.986028 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae8e08-c551-4de0-8c8d-ce2ac0740303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d864ec1fc448e2a386ae97334db442, user_id=d18e01e679fe4032bd82934f2df8e606, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:27.989308 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393810b-706c-4bac-bc58-9e878888798a None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:27.989957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae8e08-c551-4de0-8c8d-ce2ac0740303 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:27.995493 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8b1a9-6996-4512-b5d1-6ba9b5823c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:27.996449 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8b1a9-6996-4512-b5d1-6ba9b5823c3a admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:27.997232 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8b1a9-6996-4512-b5d1-6ba9b5823c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:27.997519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8b1a9-6996-4512-b5d1-6ba9b5823c3a admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:27.998339 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8b1a9-6996-4512-b5d1-6ba9b5823c3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:48:27.998718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8b1a9-6996-4512-b5d1-6ba9b5823c3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:28.000524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8b1a9-6996-4512-b5d1-6ba9b5823c3a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:28.012306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 886/1799] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:27 2022] PUT /identity/v3/projects/76d864ec1fc448e2a386ae97334db442/users/d18e01e679fe4032bd82934f2df8e606/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:48:28.020553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 914/1800] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:48:28.022562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae8e08-c551-4de0-8c8d-ce2ac0740303 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:28.025784 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f7c8c-7d6f-455a-ac7f-c6d8ae7396d4 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:28.026151 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f7c8c-7d6f-455a-ac7f-c6d8ae7396d4 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:28.026612 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f7c8c-7d6f-455a-ac7f-c6d8ae7396d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:28.063361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-4a9f7c8c-7d6f-455a-ac7f-c6d8ae7396d4 None None] MFA Rules not processed for user `d18e01e679fe4032bd82934f2df8e606`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:28.064712 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-4a9f7c8c-7d6f-455a-ac7f-c6d8ae7396d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:28.120193 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae8e08-c551-4de0-8c8d-ce2ac0740303 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:28.135599 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b85d6-e86d-478b-9ccc-eebb93621eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:28.136455 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b85d6-e86d-478b-9ccc-eebb93621eb3 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:28.136726 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b85d6-e86d-478b-9ccc-eebb93621eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:28.137615 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b85d6-e86d-478b-9ccc-eebb93621eb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:28.138344 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b85d6-e86d-478b-9ccc-eebb93621eb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:28.140458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b85d6-e86d-478b-9ccc-eebb93621eb3 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:28.212012 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 915/1801] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:28.212959 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f7c8c-7d6f-455a-ac7f-c6d8ae7396d4 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:28.215129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-4a9f7c8c-7d6f-455a-ac7f-c6d8ae7396d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:28.226509 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 887/1802] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:28 2022] POST /identity/v3/projects => generated 385 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:48:28.227844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b85d6-e86d-478b-9ccc-eebb93621eb3 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:28.230336 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-655b85d6-e86d-478b-9ccc-eebb93621eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:28.239454 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:28.368315 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f7c8c-7d6f-455a-ac7f-c6d8ae7396d4 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:28.409521 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b85d6-e86d-478b-9ccc-eebb93621eb3 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:28.420889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a5a32-d12c-48e9-9761-c9ed68fe87b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:28.421854 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a5a32-d12c-48e9-9761-c9ed68fe87b2 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:28.422140 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a5a32-d12c-48e9-9761-c9ed68fe87b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:28.422420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a5a32-d12c-48e9-9761-c9ed68fe87b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:28.481844 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0dcd89-5e93-4285-a30b-3952c7a3696b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:28.482442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dcd89-5e93-4285-a30b-3952c7a3696b service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:28.482728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dcd89-5e93-4285-a30b-3952c7a3696b service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:28.483006 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dcd89-5e93-4285-a30b-3952c7a3696b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:28.484965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-6e0dcd89-5e93-4285-a30b-3952c7a3696b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:28.565713 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a5a32-d12c-48e9-9761-c9ed68fe87b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:28.567833 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a5a32-d12c-48e9-9761-c9ed68fe87b2 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:28.579498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 888/1803] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:28.580481 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a5a32-d12c-48e9-9761-c9ed68fe87b2 service cinder] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:28.588109 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0dcd89-5e93-4285-a30b-3952c7a3696b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:28.590118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0dcd89-5e93-4285-a30b-3952c7a3696b service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:28.606113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 916/1804] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:28 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:28.607059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0dcd89-5e93-4285-a30b-3952c7a3696b service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:28.609433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:28.610585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-6e0dcd89-5e93-4285-a30b-3952c7a3696b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:28.690231 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a5a32-d12c-48e9-9761-c9ed68fe87b2 service cinder] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:28.702451 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2699d95-dbf9-476c-8d91-0f7de19d3154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:28.703345 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2699d95-dbf9-476c-8d91-0f7de19d3154 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:28.703553 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2699d95-dbf9-476c-8d91-0f7de19d3154 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:28.703752 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2699d95-dbf9-476c-8d91-0f7de19d3154 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:28.705503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2699d95-dbf9-476c-8d91-0f7de19d3154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:28.710619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2699d95-dbf9-476c-8d91-0f7de19d3154 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:28.732293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0dcd89-5e93-4285-a30b-3952c7a3696b service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:28.738959 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c58813-1c24-4c27-97da-8283a0f4c2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:28.740291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c58813-1c24-4c27-97da-8283a0f4c2f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:28.740930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c58813-1c24-4c27-97da-8283a0f4c2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:28.741409 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c58813-1c24-4c27-97da-8283a0f4c2f1 admin admin] PATH_INFO: `/v3/users/d2ecf39af75b4ac29b9a65312fd3cf12` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:28.743427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c58813-1c24-4c27-97da-8283a0f4c2f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2ecf39af75b4ac29b9a65312fd3cf12)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:28.746920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c58813-1c24-4c27-97da-8283a0f4c2f1 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:28.752813 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 889/1805] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:28 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:28.752813 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2699d95-dbf9-476c-8d91-0f7de19d3154 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:28.772181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2699d95-dbf9-476c-8d91-0f7de19d3154 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:28.773733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:28.784799 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7dd0da-85bc-4cc8-bd14-1a53da9239cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:28.789421 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7dd0da-85bc-4cc8-bd14-1a53da9239cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:28.789954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7dd0da-85bc-4cc8-bd14-1a53da9239cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:28.790438 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7dd0da-85bc-4cc8-bd14-1a53da9239cc admin admin] PATH_INFO: `/v3/users/09f525f71e1b43a2ac32c2682c323b85` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:28.792490 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7dd0da-85bc-4cc8-bd14-1a53da9239cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=09f525f71e1b43a2ac32c2682c323b85)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:28.795027 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7dd0da-85bc-4cc8-bd14-1a53da9239cc admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:28.840976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-36c58813-1c24-4c27-97da-8283a0f4c2f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2ecf39af75b4ac29b9a65312fd3cf12'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:28.864114 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-36c58813-1c24-4c27-97da-8283a0f4c2f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2ecf39af75b4ac29b9a65312fd3cf12'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:28.864637 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-6a7dd0da-85bc-4cc8-bd14-1a53da9239cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09f525f71e1b43a2ac32c2682c323b85'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:28.865217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-36c58813-1c24-4c27-97da-8283a0f4c2f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2ecf39af75b4ac29b9a65312fd3cf12'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:28.876367 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-36c58813-1c24-4c27-97da-8283a0f4c2f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2ecf39af75b4ac29b9a65312fd3cf12'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:28.887128 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-6a7dd0da-85bc-4cc8-bd14-1a53da9239cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09f525f71e1b43a2ac32c2682c323b85'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:28.887960 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-36c58813-1c24-4c27-97da-8283a0f4c2f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2ecf39af75b4ac29b9a65312fd3cf12'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:28.899742 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 917/1806] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:28 2022] DELETE /identity/v3/users/d2ecf39af75b4ac29b9a65312fd3cf12 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:28.899953 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-6a7dd0da-85bc-4cc8-bd14-1a53da9239cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09f525f71e1b43a2ac32c2682c323b85'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:28.899953 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-6a7dd0da-85bc-4cc8-bd14-1a53da9239cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09f525f71e1b43a2ac32c2682c323b85'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:28.900131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c58813-1c24-4c27-97da-8283a0f4c2f1 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:28.900304 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-6a7dd0da-85bc-4cc8-bd14-1a53da9239cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09f525f71e1b43a2ac32c2682c323b85'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:28.911595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-36c58813-1c24-4c27-97da-8283a0f4c2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:28.919012 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 890/1807] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:28 2022] DELETE /identity/v3/users/09f525f71e1b43a2ac32c2682c323b85 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:28.920834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ba9e9-48bc-440c-a9a5-845860feea89 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:28.921514 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ba9e9-48bc-440c-a9a5-845860feea89 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:28.921894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ba9e9-48bc-440c-a9a5-845860feea89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:28.953394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-5c3ba9e9-48bc-440c-a9a5-845860feea89 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:28.954728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-5c3ba9e9-48bc-440c-a9a5-845860feea89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:29.034957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 891/1808] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:29.052919 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ba9e9-48bc-440c-a9a5-845860feea89 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:29.055839 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-5c3ba9e9-48bc-440c-a9a5-845860feea89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:29.101728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c58813-1c24-4c27-97da-8283a0f4c2f1 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:29.106317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:29.109705 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b9d42-49df-4f18-91a0-68b511c3b739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:29.111526 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b9d42-49df-4f18-91a0-68b511c3b739 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:29.111948 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b9d42-49df-4f18-91a0-68b511c3b739 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:29.112317 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b9d42-49df-4f18-91a0-68b511c3b739 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:29.113405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b9d42-49df-4f18-91a0-68b511c3b739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:29.115141 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b9d42-49df-4f18-91a0-68b511c3b739 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:29.130442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 918/1809] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:29.134221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b9d42-49df-4f18-91a0-68b511c3b739 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:29.139771 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-475b9d42-49df-4f18-91a0-68b511c3b739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:29.158654 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b9d42-49df-4f18-91a0-68b511c3b739 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:29.165564 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df464e19-5578-46e1-b1f5-f07e136ffb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:29.166812 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df464e19-5578-46e1-b1f5-f07e136ffb29 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:29.167150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df464e19-5578-46e1-b1f5-f07e136ffb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:29.167464 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df464e19-5578-46e1-b1f5-f07e136ffb29 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:29.168307 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df464e19-5578-46e1-b1f5-f07e136ffb29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:29.170521 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df464e19-5578-46e1-b1f5-f07e136ffb29 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:29.192265 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 919/1810] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:48:29.198761 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ba9e9-48bc-440c-a9a5-845860feea89 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:29.201163 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df464e19-5578-46e1-b1f5-f07e136ffb29 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:29.202248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc2216-6e7b-493b-b1fa-50006a17655c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:29.202926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc2216-6e7b-493b-b1fa-50006a17655c service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:29.203215 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc2216-6e7b-493b-b1fa-50006a17655c service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:29.203545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc2216-6e7b-493b-b1fa-50006a17655c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:29.214573 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc2216-6e7b-493b-b1fa-50006a17655c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:29.216045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc2216-6e7b-493b-b1fa-50006a17655c service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:29.227940 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df464e19-5578-46e1-b1f5-f07e136ffb29 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:29.233811 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 892/1811] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:29.235027 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc2216-6e7b-493b-b1fa-50006a17655c service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:29.237729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-47fc2216-6e7b-493b-b1fa-50006a17655c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:29.238572 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3363758f-59ac-43d1-9def-c08a66a5737d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:29.239750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363758f-59ac-43d1-9def-c08a66a5737d admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:29.240077 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363758f-59ac-43d1-9def-c08a66a5737d admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:29.240788 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363758f-59ac-43d1-9def-c08a66a5737d admin admin] PATH_INFO: `/v3/projects/1fa9bbae31c44721acbbb57f6d2a3de0/users/83936890f89f4073999f1643874c3c0c/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:29.265829 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3363758f-59ac-43d1-9def-c08a66a5737d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fa9bbae31c44721acbbb57f6d2a3de0, user_id=83936890f89f4073999f1643874c3c0c, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:29.266793 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc2216-6e7b-493b-b1fa-50006a17655c service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:29.268006 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3363758f-59ac-43d1-9def-c08a66a5737d admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:29.275744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6b858-75bf-45b3-970d-a2ab116c1934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:29.277249 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6b858-75bf-45b3-970d-a2ab116c1934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:29.277534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6b858-75bf-45b3-970d-a2ab116c1934 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:29.277933 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6b858-75bf-45b3-970d-a2ab116c1934 admin admin] PATH_INFO: `/v3/projects/7dbc67b63ef442bc9731b67b8f2eff96` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:29.279475 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6b858-75bf-45b3-970d-a2ab116c1934 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dbc67b63ef442bc9731b67b8f2eff96)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:29.281504 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6b858-75bf-45b3-970d-a2ab116c1934 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:29.285201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 920/1812] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:29 2022] PUT /identity/v3/projects/1fa9bbae31c44721acbbb57f6d2a3de0/users/83936890f89f4073999f1643874c3c0c/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:29.297132 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c3a18-59f6-4b64-9ded-5e0f090bb071 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:29.297440 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c3a18-59f6-4b64-9ded-5e0f090bb071 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:29.297744 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c3a18-59f6-4b64-9ded-5e0f090bb071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:29.329867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-f99c3a18-59f6-4b64-9ded-5e0f090bb071 None None] MFA Rules not processed for user `83936890f89f4073999f1643874c3c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:29.332169 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f99c3a18-59f6-4b64-9ded-5e0f090bb071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:29.354698 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-afd6b858-75bf-45b3-970d-a2ab116c1934 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dbc67b63ef442bc9731b67b8f2eff96', 'request_id': 'req-afd6b858-75bf-45b3-970d-a2ab116c1934'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.362489 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-afd6b858-75bf-45b3-970d-a2ab116c1934 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dbc67b63ef442bc9731b67b8f2eff96', 'request_id': 'req-afd6b858-75bf-45b3-970d-a2ab116c1934'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.368870 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-afd6b858-75bf-45b3-970d-a2ab116c1934 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dbc67b63ef442bc9731b67b8f2eff96', 'request_id': 'req-afd6b858-75bf-45b3-970d-a2ab116c1934'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.385744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-afd6b858-75bf-45b3-970d-a2ab116c1934 admin admin] The token cache is being invalidate because project 7dbc67b63ef442bc9731b67b8f2eff96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:29.386768 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-afd6b858-75bf-45b3-970d-a2ab116c1934 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.387530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-afd6b858-75bf-45b3-970d-a2ab116c1934 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.390363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:29.390486 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 893/1813] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:29 2022] DELETE /identity/v3/projects/7dbc67b63ef442bc9731b67b8f2eff96 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:29.395147 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6b858-75bf-45b3-970d-a2ab116c1934 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:29.398348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-afd6b858-75bf-45b3-970d-a2ab116c1934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:29.434416 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 921/1814] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:48:29.439296 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c3a18-59f6-4b64-9ded-5e0f090bb071 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:29.441925 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f99c3a18-59f6-4b64-9ded-5e0f090bb071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:29.499817 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6b858-75bf-45b3-970d-a2ab116c1934 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:29.505512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3037b-da83-49ec-b7d3-5b14c291f4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:29.506187 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3037b-da83-49ec-b7d3-5b14c291f4f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:29.506991 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3037b-da83-49ec-b7d3-5b14c291f4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:29.507411 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3037b-da83-49ec-b7d3-5b14c291f4f1 admin admin] PATH_INFO: `/v3/users/acd9d9dd27ee492ea23f889e3a8114f8` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:29.508843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b3037b-da83-49ec-b7d3-5b14c291f4f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acd9d9dd27ee492ea23f889e3a8114f8)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:29.510836 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b3037b-da83-49ec-b7d3-5b14c291f4f1 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:29.540685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c3a18-59f6-4b64-9ded-5e0f090bb071 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:29.547020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0346ca09-4093-435a-b35c-efc8294a3c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:29.548180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0346ca09-4093-435a-b35c-efc8294a3c62 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:29.548715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0346ca09-4093-435a-b35c-efc8294a3c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:29.549003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0346ca09-4093-435a-b35c-efc8294a3c62 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:29.549934 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0346ca09-4093-435a-b35c-efc8294a3c62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:29.552451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0346ca09-4093-435a-b35c-efc8294a3c62 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:29.568325 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:29.571456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-90b3037b-da83-49ec-b7d3-5b14c291f4f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acd9d9dd27ee492ea23f889e3a8114f8'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.589788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-90b3037b-da83-49ec-b7d3-5b14c291f4f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acd9d9dd27ee492ea23f889e3a8114f8'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.597864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 922/1815] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:29 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:48:29.599812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-90b3037b-da83-49ec-b7d3-5b14c291f4f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acd9d9dd27ee492ea23f889e3a8114f8'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.600591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-90b3037b-da83-49ec-b7d3-5b14c291f4f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acd9d9dd27ee492ea23f889e3a8114f8'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.601582 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-90b3037b-da83-49ec-b7d3-5b14c291f4f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acd9d9dd27ee492ea23f889e3a8114f8'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.602607 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0346ca09-4093-435a-b35c-efc8294a3c62 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:29.604417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0346ca09-4093-435a-b35c-efc8294a3c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:29.620279 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 894/1816] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:29 2022] DELETE /identity/v3/users/acd9d9dd27ee492ea23f889e3a8114f8 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:29.622013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3037b-da83-49ec-b7d3-5b14c291f4f1 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:29.631006 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-90b3037b-da83-49ec-b7d3-5b14c291f4f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:29.695349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0346ca09-4093-435a-b35c-efc8294a3c62 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:29.700969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ada127-f525-4368-97c4-0530549a1940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:29.701682 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ada127-f525-4368-97c4-0530549a1940 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:29.701978 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ada127-f525-4368-97c4-0530549a1940 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:29.702262 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ada127-f525-4368-97c4-0530549a1940 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:29.702965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ada127-f525-4368-97c4-0530549a1940 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:29.704910 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ada127-f525-4368-97c4-0530549a1940 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:29.712230 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3037b-da83-49ec-b7d3-5b14c291f4f1 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:29.717528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd4d87-4d54-40ac-a728-c9b7d4cf5816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:29.718621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd4d87-4d54-40ac-a728-c9b7d4cf5816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:29.718995 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd4d87-4d54-40ac-a728-c9b7d4cf5816 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:29.719360 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd4d87-4d54-40ac-a728-c9b7d4cf5816 admin admin] PATH_INFO: `/v3/projects/f6ebada18659446ca3d1ca54af3cd71a` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:29.720683 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd4d87-4d54-40ac-a728-c9b7d4cf5816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ebada18659446ca3d1ca54af3cd71a)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:29.722821 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd4d87-4d54-40ac-a728-c9b7d4cf5816 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:29.730150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 923/1817] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:29 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:29.733414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ada127-f525-4368-97c4-0530549a1940 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:29.744515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ada127-f525-4368-97c4-0530549a1940 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:29.749512 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51009728-8ec8-45ba-8166-47de31bf9df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:29.750347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51009728-8ec8-45ba-8166-47de31bf9df6 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:29.750682 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51009728-8ec8-45ba-8166-47de31bf9df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:29.750842 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51009728-8ec8-45ba-8166-47de31bf9df6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:29.751584 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51009728-8ec8-45ba-8166-47de31bf9df6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:29.753390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51009728-8ec8-45ba-8166-47de31bf9df6 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:29.765364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 924/1818] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:29.770652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51009728-8ec8-45ba-8166-47de31bf9df6 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:29.783863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51009728-8ec8-45ba-8166-47de31bf9df6 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:29.789381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-6bfd4d87-4d54-40ac-a728-c9b7d4cf5816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ebada18659446ca3d1ca54af3cd71a', 'request_id': 'req-6bfd4d87-4d54-40ac-a728-c9b7d4cf5816'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.803662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-6bfd4d87-4d54-40ac-a728-c9b7d4cf5816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ebada18659446ca3d1ca54af3cd71a', 'request_id': 'req-6bfd4d87-4d54-40ac-a728-c9b7d4cf5816'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.813409 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-6bfd4d87-4d54-40ac-a728-c9b7d4cf5816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ebada18659446ca3d1ca54af3cd71a', 'request_id': 'req-6bfd4d87-4d54-40ac-a728-c9b7d4cf5816'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.833913 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-6bfd4d87-4d54-40ac-a728-c9b7d4cf5816 admin admin] The token cache is being invalidate because project f6ebada18659446ca3d1ca54af3cd71a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:29.834213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-6bfd4d87-4d54-40ac-a728-c9b7d4cf5816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.835084 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-6bfd4d87-4d54-40ac-a728-c9b7d4cf5816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.837065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 895/1819] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:29 2022] DELETE /identity/v3/projects/f6ebada18659446ca3d1ca54af3cd71a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:29.841182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd4d87-4d54-40ac-a728-c9b7d4cf5816 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:29.842578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-6bfd4d87-4d54-40ac-a728-c9b7d4cf5816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:29.846261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:29.865404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382031ea-4e93-489d-90bf-b403b1488ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:29.867390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382031ea-4e93-489d-90bf-b403b1488ebf admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:29.867659 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382031ea-4e93-489d-90bf-b403b1488ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:29.867959 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382031ea-4e93-489d-90bf-b403b1488ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:29.868764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382031ea-4e93-489d-90bf-b403b1488ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:29.870410 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382031ea-4e93-489d-90bf-b403b1488ebf admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:29.881059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 925/1820] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:29.881059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382031ea-4e93-489d-90bf-b403b1488ebf admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:29.881059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-382031ea-4e93-489d-90bf-b403b1488ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:29.898249 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382031ea-4e93-489d-90bf-b403b1488ebf admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:29.903886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd6354-ef84-4048-a7a4-2c138a2e2fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:29.904797 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd6354-ef84-4048-a7a4-2c138a2e2fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:29.905083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd6354-ef84-4048-a7a4-2c138a2e2fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:29.905369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd6354-ef84-4048-a7a4-2c138a2e2fe3 admin admin] PATH_INFO: `/v3/projects/2e63eb9f6ae448ee82a2e6a24001bfc9` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:29.906610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd6354-ef84-4048-a7a4-2c138a2e2fe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e63eb9f6ae448ee82a2e6a24001bfc9)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:29.908531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd6354-ef84-4048-a7a4-2c138a2e2fe3 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:29.908765 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:29.924632 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd4d87-4d54-40ac-a728-c9b7d4cf5816 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:29.932163 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331299e0-cf27-42ca-a789-4f7112426846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:29.933489 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331299e0-cf27-42ca-a789-4f7112426846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:29.933823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331299e0-cf27-42ca-a789-4f7112426846 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:29.934105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331299e0-cf27-42ca-a789-4f7112426846 admin admin] PATH_INFO: `/v3/users/d18e01e679fe4032bd82934f2df8e606` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:29.937130 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331299e0-cf27-42ca-a789-4f7112426846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d18e01e679fe4032bd82934f2df8e606)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:29.939769 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331299e0-cf27-42ca-a789-4f7112426846 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:29.962010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4afd6354-ef84-4048-a7a4-2c138a2e2fe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e63eb9f6ae448ee82a2e6a24001bfc9', 'request_id': 'req-4afd6354-ef84-4048-a7a4-2c138a2e2fe3'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.970150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4afd6354-ef84-4048-a7a4-2c138a2e2fe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e63eb9f6ae448ee82a2e6a24001bfc9', 'request_id': 'req-4afd6354-ef84-4048-a7a4-2c138a2e2fe3'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.980014 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4afd6354-ef84-4048-a7a4-2c138a2e2fe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e63eb9f6ae448ee82a2e6a24001bfc9', 'request_id': 'req-4afd6354-ef84-4048-a7a4-2c138a2e2fe3'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.992531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4afd6354-ef84-4048-a7a4-2c138a2e2fe3 admin admin] The token cache is being invalidate because project 2e63eb9f6ae448ee82a2e6a24001bfc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:29.992802 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4afd6354-ef84-4048-a7a4-2c138a2e2fe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.993480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4afd6354-ef84-4048-a7a4-2c138a2e2fe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.995141 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 926/1821] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:29 2022] DELETE /identity/v3/projects/2e63eb9f6ae448ee82a2e6a24001bfc9 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:29.998947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd6354-ef84-4048-a7a4-2c138a2e2fe3 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:29.999058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-331299e0-cf27-42ca-a789-4f7112426846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd18e01e679fe4032bd82934f2df8e606'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:29.999144 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-4afd6354-ef84-4048-a7a4-2c138a2e2fe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:30.018854 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-331299e0-cf27-42ca-a789-4f7112426846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd18e01e679fe4032bd82934f2df8e606'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:30.028336 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-331299e0-cf27-42ca-a789-4f7112426846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd18e01e679fe4032bd82934f2df8e606'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:30.029131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-331299e0-cf27-42ca-a789-4f7112426846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd18e01e679fe4032bd82934f2df8e606'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:30.029859 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-331299e0-cf27-42ca-a789-4f7112426846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd18e01e679fe4032bd82934f2df8e606'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:30.044345 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 896/1822] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:29 2022] DELETE /identity/v3/users/d18e01e679fe4032bd82934f2df8e606 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:48:30.045861 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f83c5-cb5c-431b-b651-7bf3b63e8cd3 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:30.046403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f83c5-cb5c-431b-b651-7bf3b63e8cd3 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:30.046695 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f83c5-cb5c-431b-b651-7bf3b63e8cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:30.065876 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-713f83c5-cb5c-431b-b651-7bf3b63e8cd3 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:30.067118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-713f83c5-cb5c-431b-b651-7bf3b63e8cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:30.084982 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd6354-ef84-4048-a7a4-2c138a2e2fe3 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:30.145212 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 897/1823] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:30.155074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffbc33-53bb-4c51-8109-a8425d3d3666 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:30.155418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffbc33-53bb-4c51-8109-a8425d3d3666 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:30.155704 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffbc33-53bb-4c51-8109-a8425d3d3666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:30.180966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-81ffbc33-53bb-4c51-8109-a8425d3d3666 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:30.181247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0378fb-ff4b-4049-af99-a77bad429b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:30.181716 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-81ffbc33-53bb-4c51-8109-a8425d3d3666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:30.182702 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0378fb-ff4b-4049-af99-a77bad429b8d admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:30.183252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0378fb-ff4b-4049-af99-a77bad429b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:30.183545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0378fb-ff4b-4049-af99-a77bad429b8d admin admin] PATH_INFO: `/v3/projects/0b7483726a8546188eb3b7ae221ca8da/users/c071987e60d4429dbfb2c9a587461044/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:30.195107 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 898/1824] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:48:30.197111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffbc33-53bb-4c51-8109-a8425d3d3666 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:30.208091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffbc33-53bb-4c51-8109-a8425d3d3666 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:30.209929 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0378fb-ff4b-4049-af99-a77bad429b8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b7483726a8546188eb3b7ae221ca8da, user_id=c071987e60d4429dbfb2c9a587461044, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:30.212671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0378fb-ff4b-4049-af99-a77bad429b8d admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:30.216803 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:30.220150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2360e4-e6e4-42fd-8c8a-8abeaaddfed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:30.222339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2360e4-e6e4-42fd-8c8a-8abeaaddfed1 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:30.222645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2360e4-e6e4-42fd-8c8a-8abeaaddfed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:30.222940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2360e4-e6e4-42fd-8c8a-8abeaaddfed1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:30.223694 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2360e4-e6e4-42fd-8c8a-8abeaaddfed1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:48:30.224063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2360e4-e6e4-42fd-8c8a-8abeaaddfed1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:30.226418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2360e4-e6e4-42fd-8c8a-8abeaaddfed1 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:30.244247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 927/1825] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:29 2022] PUT /identity/v3/projects/0b7483726a8546188eb3b7ae221ca8da/users/c071987e60d4429dbfb2c9a587461044/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:30.244468 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 899/1826] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:48:30.246576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0378fb-ff4b-4049-af99-a77bad429b8d admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:30.254735 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe72f7-a4d3-4cc5-8aaa-fe02789a0761 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:30.254961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe72f7-a4d3-4cc5-8aaa-fe02789a0761 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:30.255249 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe72f7-a4d3-4cc5-8aaa-fe02789a0761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:30.302103 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-fbbe72f7-a4d3-4cc5-8aaa-fe02789a0761 None None] MFA Rules not processed for user `c071987e60d4429dbfb2c9a587461044`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:30.303474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-fbbe72f7-a4d3-4cc5-8aaa-fe02789a0761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:30.328694 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0378fb-ff4b-4049-af99-a77bad429b8d admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:30.335981 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b93fb2-e972-4ae3-9d53-077cece208b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:30.336429 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b93fb2-e972-4ae3-9d53-077cece208b2 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:30.336766 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b93fb2-e972-4ae3-9d53-077cece208b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:30.337178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b93fb2-e972-4ae3-9d53-077cece208b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:30.338051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b93fb2-e972-4ae3-9d53-077cece208b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:30.339889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b93fb2-e972-4ae3-9d53-077cece208b2 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:30.369974 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:30.372268 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 928/1827] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:30 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:30.377588 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b93fb2-e972-4ae3-9d53-077cece208b2 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:30.414238 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 900/1828] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:48:30.418479 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe72f7-a4d3-4cc5-8aaa-fe02789a0761 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:30.420057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-fbbe72f7-a4d3-4cc5-8aaa-fe02789a0761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:30.436811 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe72f7-a4d3-4cc5-8aaa-fe02789a0761 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:30.464082 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b93fb2-e972-4ae3-9d53-077cece208b2 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:30.469111 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dd459-74e4-4964-8889-ea8d85abdf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:30.469847 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dd459-74e4-4964-8889-ea8d85abdf30 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:30.470134 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dd459-74e4-4964-8889-ea8d85abdf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:30.470413 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dd459-74e4-4964-8889-ea8d85abdf30 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:30.473135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4dd459-74e4-4964-8889-ea8d85abdf30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:30.475202 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4dd459-74e4-4964-8889-ea8d85abdf30 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:30.505767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 929/1829] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:30 2022] POST /identity/v3/users => generated 291 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:30.508325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dd459-74e4-4964-8889-ea8d85abdf30 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:30.518112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dd459-74e4-4964-8889-ea8d85abdf30 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:30.520996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3ffbe-7692-4a0f-8e64-afb790d0f53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:30.521876 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3ffbe-7692-4a0f-8e64-afb790d0f53a admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:30.522235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3ffbe-7692-4a0f-8e64-afb790d0f53a admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:30.522523 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3ffbe-7692-4a0f-8e64-afb790d0f53a admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:30.523392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3ffbe-7692-4a0f-8e64-afb790d0f53a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:30.525783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e295715-9e87-44a1-81e8-a74e90c5264e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:30.526670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e295715-9e87-44a1-81e8-a74e90c5264e admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:30.526824 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e295715-9e87-44a1-81e8-a74e90c5264e admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:30.527131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e295715-9e87-44a1-81e8-a74e90c5264e admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:30.527548 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3ffbe-7692-4a0f-8e64-afb790d0f53a admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:30.528437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e295715-9e87-44a1-81e8-a74e90c5264e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:30.530433 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e295715-9e87-44a1-81e8-a74e90c5264e admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:30.544769 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 930/1830] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:30.545741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e295715-9e87-44a1-81e8-a74e90c5264e admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:30.549591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-6e295715-9e87-44a1-81e8-a74e90c5264e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:30.560352 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 901/1831] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:30 2022] POST /identity/v3/projects => generated 387 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:48:30.561349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3ffbe-7692-4a0f-8e64-afb790d0f53a admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:30.567013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e295715-9e87-44a1-81e8-a74e90c5264e admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:30.639101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3ffbe-7692-4a0f-8e64-afb790d0f53a admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:30.643739 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe53ba-61ce-449d-ba38-f2f4b437fca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:30.644079 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd40585-0cc3-40f8-a781-4fc7f7e0357f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:30.644767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe53ba-61ce-449d-ba38-f2f4b437fca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:30.644877 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd40585-0cc3-40f8-a781-4fc7f7e0357f admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:30.645052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe53ba-61ce-449d-ba38-f2f4b437fca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:30.645275 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd40585-0cc3-40f8-a781-4fc7f7e0357f admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:30.645376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe53ba-61ce-449d-ba38-f2f4b437fca4 admin admin] PATH_INFO: `/v3/projects/76d864ec1fc448e2a386ae97334db442` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:30.645933 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd40585-0cc3-40f8-a781-4fc7f7e0357f admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:30.646586 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe53ba-61ce-449d-ba38-f2f4b437fca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d864ec1fc448e2a386ae97334db442)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:30.646707 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd40585-0cc3-40f8-a781-4fc7f7e0357f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:30.648957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd40585-0cc3-40f8-a781-4fc7f7e0357f admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:30.649113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe53ba-61ce-449d-ba38-f2f4b437fca4 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:30.650782 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:30.660660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 902/1832] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:30.661550 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd40585-0cc3-40f8-a781-4fc7f7e0357f admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:30.673904 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd40585-0cc3-40f8-a781-4fc7f7e0357f admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:30.683489 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6ab53-42c2-4071-8660-1b0a0693b5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:30.684242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6ab53-42c2-4071-8660-1b0a0693b5ef admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:30.684596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6ab53-42c2-4071-8660-1b0a0693b5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:30.684942 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6ab53-42c2-4071-8660-1b0a0693b5ef admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:30.685755 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6ab53-42c2-4071-8660-1b0a0693b5ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:30.687958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6ab53-42c2-4071-8660-1b0a0693b5ef admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:30.708859 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-0dfe53ba-61ce-449d-ba38-f2f4b437fca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d864ec1fc448e2a386ae97334db442', 'request_id': 'req-0dfe53ba-61ce-449d-ba38-f2f4b437fca4'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:30.709573 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 903/1833] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:30 2022] POST /identity/v3/users => generated 300 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:30.710718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6ab53-42c2-4071-8660-1b0a0693b5ef admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:30.716472 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-0dfe53ba-61ce-449d-ba38-f2f4b437fca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d864ec1fc448e2a386ae97334db442', 'request_id': 'req-0dfe53ba-61ce-449d-ba38-f2f4b437fca4'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:30.720577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6ab53-42c2-4071-8660-1b0a0693b5ef admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:30.724662 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-0dfe53ba-61ce-449d-ba38-f2f4b437fca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d864ec1fc448e2a386ae97334db442', 'request_id': 'req-0dfe53ba-61ce-449d-ba38-f2f4b437fca4'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:30.738600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-0dfe53ba-61ce-449d-ba38-f2f4b437fca4 admin admin] The token cache is being invalidate because project 76d864ec1fc448e2a386ae97334db442 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:30.739240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-0dfe53ba-61ce-449d-ba38-f2f4b437fca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:30.739682 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-0dfe53ba-61ce-449d-ba38-f2f4b437fca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:30.741393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 931/1834] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:30 2022] DELETE /identity/v3/projects/76d864ec1fc448e2a386ae97334db442 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:30.742209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe53ba-61ce-449d-ba38-f2f4b437fca4 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:30.743577 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0dfe53ba-61ce-449d-ba38-f2f4b437fca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:30.768235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:30.784156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94ff7a-090c-4052-8976-7b1252a25be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:30.785059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94ff7a-090c-4052-8976-7b1252a25be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:30.785505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94ff7a-090c-4052-8976-7b1252a25be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:30.785808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94ff7a-090c-4052-8976-7b1252a25be0 admin admin] PATH_INFO: `/v3/projects/a06d17b0a34443889304dd82a1148821/users/df380f002e294de3b8aaaa80f6b8e9a3/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:30.801916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94ff7a-090c-4052-8976-7b1252a25be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a06d17b0a34443889304dd82a1148821, user_id=df380f002e294de3b8aaaa80f6b8e9a3, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:30.804643 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94ff7a-090c-4052-8976-7b1252a25be0 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:30.818016 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe53ba-61ce-449d-ba38-f2f4b437fca4 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:30.822379 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 904/1835] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:30 2022] PUT /identity/v3/projects/a06d17b0a34443889304dd82a1148821/users/df380f002e294de3b8aaaa80f6b8e9a3/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:30.824676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2f1f3-fbaf-43ba-96a7-1bdf40bca670 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:30.825196 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2f1f3-fbaf-43ba-96a7-1bdf40bca670 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:30.825762 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2f1f3-fbaf-43ba-96a7-1bdf40bca670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:30.851040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-75f2f1f3-fbaf-43ba-96a7-1bdf40bca670 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:30.852198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-75f2f1f3-fbaf-43ba-96a7-1bdf40bca670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:30.889433 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38c6d7-eee2-4ec4-92b9-b8af4d3998e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:30.890208 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38c6d7-eee2-4ec4-92b9-b8af4d3998e1 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:30.890512 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38c6d7-eee2-4ec4-92b9-b8af4d3998e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:30.890787 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38c6d7-eee2-4ec4-92b9-b8af4d3998e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:30.891590 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38c6d7-eee2-4ec4-92b9-b8af4d3998e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:30.893241 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38c6d7-eee2-4ec4-92b9-b8af4d3998e1 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:30.904052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 932/1836] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:30.905350 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61bb2e-efc7-43d5-8a5e-9e36c8e436e8 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:30.905666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61bb2e-efc7-43d5-8a5e-9e36c8e436e8 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:30.905944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61bb2e-efc7-43d5-8a5e-9e36c8e436e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:30.919326 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 905/1837] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:30.920368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2f1f3-fbaf-43ba-96a7-1bdf40bca670 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:30.931658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2f1f3-fbaf-43ba-96a7-1bdf40bca670 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:30.933963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-0f61bb2e-efc7-43d5-8a5e-9e36c8e436e8 None None] MFA Rules not processed for user `df380f002e294de3b8aaaa80f6b8e9a3`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:30.935239 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0f61bb2e-efc7-43d5-8a5e-9e36c8e436e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:30.936611 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110169b9-4bcd-46c2-b045-29db8cd8943b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:30.937506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110169b9-4bcd-46c2-b045-29db8cd8943b admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:30.937976 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110169b9-4bcd-46c2-b045-29db8cd8943b admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:30.938267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110169b9-4bcd-46c2-b045-29db8cd8943b admin admin] PATH_INFO: `/v3/projects/1e4ae28c23cf4df19940018b5e6d4a6b/users/2adde4db87ba473a9dfb5b2078f18eb5/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:30.954659 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110169b9-4bcd-46c2-b045-29db8cd8943b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4ae28c23cf4df19940018b5e6d4a6b, user_id=2adde4db87ba473a9dfb5b2078f18eb5, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:30.957286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:30.957431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110169b9-4bcd-46c2-b045-29db8cd8943b admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:30.972526 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 906/1838] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:30 2022] PUT /identity/v3/projects/1e4ae28c23cf4df19940018b5e6d4a6b/users/2adde4db87ba473a9dfb5b2078f18eb5/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:30.973917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e29fc-cfa8-4c7c-853d-02f0a0d78c92 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:30.974205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e29fc-cfa8-4c7c-853d-02f0a0d78c92 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:30.974480 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e29fc-cfa8-4c7c-853d-02f0a0d78c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:30.996314 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-819e29fc-cfa8-4c7c-853d-02f0a0d78c92 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:30.997532 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-819e29fc-cfa8-4c7c-853d-02f0a0d78c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:31.045589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 933/1839] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:31.046918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61bb2e-efc7-43d5-8a5e-9e36c8e436e8 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:31.062639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61bb2e-efc7-43d5-8a5e-9e36c8e436e8 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:31.074921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 907/1840] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:31.076147 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e29fc-cfa8-4c7c-853d-02f0a0d78c92 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:31.079830 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-819e29fc-cfa8-4c7c-853d-02f0a0d78c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:31.093404 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e29fc-cfa8-4c7c-853d-02f0a0d78c92 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:31.098851 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d9dc2-81f9-4a4b-a5e2-88648dcffac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:31.099683 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d9dc2-81f9-4a4b-a5e2-88648dcffac1 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:31.100457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d9dc2-81f9-4a4b-a5e2-88648dcffac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:31.100748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d9dc2-81f9-4a4b-a5e2-88648dcffac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:31.101550 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d9dc2-81f9-4a4b-a5e2-88648dcffac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:31.103691 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d9dc2-81f9-4a4b-a5e2-88648dcffac1 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:31.137730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 908/1841] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:31 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:31.138698 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d9dc2-81f9-4a4b-a5e2-88648dcffac1 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:31.157378 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d95b7-7d5c-4f59-bfb6-2434fc0d44b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:31.158144 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d95b7-7d5c-4f59-bfb6-2434fc0d44b6 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:31.158431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d95b7-7d5c-4f59-bfb6-2434fc0d44b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:31.158709 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d95b7-7d5c-4f59-bfb6-2434fc0d44b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:31.159556 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d95b7-7d5c-4f59-bfb6-2434fc0d44b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:31.161324 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d95b7-7d5c-4f59-bfb6-2434fc0d44b6 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:31.173576 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:31.175653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 934/1842] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:31.176571 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d95b7-7d5c-4f59-bfb6-2434fc0d44b6 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:31.192692 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d95b7-7d5c-4f59-bfb6-2434fc0d44b6 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:31.232355 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d9dc2-81f9-4a4b-a5e2-88648dcffac1 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:31.239583 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfe613-524c-41d3-9ef5-29558776caca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:31.240373 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfe613-524c-41d3-9ef5-29558776caca admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:31.241193 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfe613-524c-41d3-9ef5-29558776caca admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:31.242168 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfe613-524c-41d3-9ef5-29558776caca admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:31.243411 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfe613-524c-41d3-9ef5-29558776caca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:48:31.243754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfe613-524c-41d3-9ef5-29558776caca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:31.249902 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfe613-524c-41d3-9ef5-29558776caca admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:31.293288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 909/1843] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:48:31.294758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfe613-524c-41d3-9ef5-29558776caca admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:31.315941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfe613-524c-41d3-9ef5-29558776caca admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:31.316145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f42ccf-7d22-49f8-9395-e2b4666dc614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:31.317189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f42ccf-7d22-49f8-9395-e2b4666dc614 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:31.317492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f42ccf-7d22-49f8-9395-e2b4666dc614 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:31.317809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f42ccf-7d22-49f8-9395-e2b4666dc614 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:31.318656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f42ccf-7d22-49f8-9395-e2b4666dc614 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:31.321289 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f42ccf-7d22-49f8-9395-e2b4666dc614 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:31.321415 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322c1aa-495e-4730-b1e9-8ec6bd8a496b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:31.322354 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322c1aa-495e-4730-b1e9-8ec6bd8a496b admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:31.322731 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322c1aa-495e-4730-b1e9-8ec6bd8a496b admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:31.323014 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322c1aa-495e-4730-b1e9-8ec6bd8a496b admin admin] PATH_INFO: `/v3/projects/1e4ae28c23cf4df19940018b5e6d4a6b/users/2adde4db87ba473a9dfb5b2078f18eb5/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:31.325043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322c1aa-495e-4730-b1e9-8ec6bd8a496b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4ae28c23cf4df19940018b5e6d4a6b, user_id=2adde4db87ba473a9dfb5b2078f18eb5, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:31.327721 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322c1aa-495e-4730-b1e9-8ec6bd8a496b admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:31.345021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 910/1844] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:31 2022] PUT /identity/v3/projects/1e4ae28c23cf4df19940018b5e6d4a6b/users/2adde4db87ba473a9dfb5b2078f18eb5/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:31.348483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 935/1845] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:31 2022] POST /identity/v3/users => generated 296 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:31.349979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f42ccf-7d22-49f8-9395-e2b4666dc614 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:31.356930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322c1aa-495e-4730-b1e9-8ec6bd8a496b admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:31.363477 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f42ccf-7d22-49f8-9395-e2b4666dc614 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:31.442032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322c1aa-495e-4730-b1e9-8ec6bd8a496b admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:31.451780 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:31.454207 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b5058-af4b-42cd-87b3-1b64dca314c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:31.456281 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b5058-af4b-42cd-87b3-1b64dca314c1 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:31.456644 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b5058-af4b-42cd-87b3-1b64dca314c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:31.457112 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b5058-af4b-42cd-87b3-1b64dca314c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:31.458572 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b5058-af4b-42cd-87b3-1b64dca314c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:31.465738 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b5058-af4b-42cd-87b3-1b64dca314c1 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:31.488642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d3808-f890-4232-aaf5-017e06241f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:31.490005 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d3808-f890-4232-aaf5-017e06241f5f admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:31.490485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d3808-f890-4232-aaf5-017e06241f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:31.490970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d3808-f890-4232-aaf5-017e06241f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:31.492307 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d3808-f890-4232-aaf5-017e06241f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:31.494398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d3808-f890-4232-aaf5-017e06241f5f admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:31.506816 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 911/1846] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:31 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:31.509546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 936/1847] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:48:31.509770 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b3191-61d2-4f87-a651-3f82cd0306f3 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:31.510299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b3191-61d2-4f87-a651-3f82cd0306f3 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:31.510716 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b3191-61d2-4f87-a651-3f82cd0306f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:31.512253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d3808-f890-4232-aaf5-017e06241f5f admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:31.535015 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d3808-f890-4232-aaf5-017e06241f5f admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:31.552281 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-7e7b3191-61d2-4f87-a651-3f82cd0306f3 None None] MFA Rules not processed for user `2adde4db87ba473a9dfb5b2078f18eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:31.554531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-7e7b3191-61d2-4f87-a651-3f82cd0306f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:31.652759 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96596e35-a268-46be-8903-21abedfad0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:31.653495 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96596e35-a268-46be-8903-21abedfad0d0 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:31.654206 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96596e35-a268-46be-8903-21abedfad0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:31.654489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96596e35-a268-46be-8903-21abedfad0d0 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:31.655223 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96596e35-a268-46be-8903-21abedfad0d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:31.657426 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96596e35-a268-46be-8903-21abedfad0d0 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:31.688030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 937/1848] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:31 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:31.689028 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96596e35-a268-46be-8903-21abedfad0d0 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:31.714391 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96596e35-a268-46be-8903-21abedfad0d0 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:31.732149 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c4263-c900-40e8-83b4-d9cd348b18c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:31.733077 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c4263-c900-40e8-83b4-d9cd348b18c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:31.733367 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c4263-c900-40e8-83b4-d9cd348b18c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:31.733673 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c4263-c900-40e8-83b4-d9cd348b18c4 admin admin] PATH_INFO: `/v3/projects/3984f694f3fe41bf834deb75418d5230/users/7dfac160848c4fb59454585defeed3d2/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:31.775272 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c4263-c900-40e8-83b4-d9cd348b18c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3984f694f3fe41bf834deb75418d5230, user_id=7dfac160848c4fb59454585defeed3d2, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:31.778157 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c4263-c900-40e8-83b4-d9cd348b18c4 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:31.801912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 938/1849] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:31 2022] PUT /identity/v3/projects/3984f694f3fe41bf834deb75418d5230/users/7dfac160848c4fb59454585defeed3d2/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:31.801912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c4263-c900-40e8-83b4-d9cd348b18c4 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:31.888273 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c4263-c900-40e8-83b4-d9cd348b18c4 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:31.894772 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db57a6ac-e0d1-4f02-ad3a-26bdefe1271d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:31.895227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57a6ac-e0d1-4f02-ad3a-26bdefe1271d admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:31.895516 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57a6ac-e0d1-4f02-ad3a-26bdefe1271d admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:31.895796 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57a6ac-e0d1-4f02-ad3a-26bdefe1271d admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:31.896668 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db57a6ac-e0d1-4f02-ad3a-26bdefe1271d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:31.899345 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db57a6ac-e0d1-4f02-ad3a-26bdefe1271d admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:31.900638 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 912/1850] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:31.901731 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b3191-61d2-4f87-a651-3f82cd0306f3 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:31.918154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 939/1851] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:31.918154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db57a6ac-e0d1-4f02-ad3a-26bdefe1271d admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:31.918497 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-db57a6ac-e0d1-4f02-ad3a-26bdefe1271d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:31.936396 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b3191-61d2-4f87-a651-3f82cd0306f3 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:31.954512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc598e18-0c20-42eb-9ccd-743d09fb530b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:31.955932 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc598e18-0c20-42eb-9ccd-743d09fb530b admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:31.956231 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc598e18-0c20-42eb-9ccd-743d09fb530b admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:31.956590 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc598e18-0c20-42eb-9ccd-743d09fb530b admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:31.957848 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc598e18-0c20-42eb-9ccd-743d09fb530b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:31.959855 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc598e18-0c20-42eb-9ccd-743d09fb530b admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:31.974068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 913/1852] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:31.974543 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc598e18-0c20-42eb-9ccd-743d09fb530b admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:31.984435 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:31.991887 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc598e18-0c20-42eb-9ccd-743d09fb530b admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:32.000063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d20009-4048-4b24-8a13-d9bb34b5ca7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:32.000755 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d20009-4048-4b24-8a13-d9bb34b5ca7c admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:32.001127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d20009-4048-4b24-8a13-d9bb34b5ca7c admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:32.001452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d20009-4048-4b24-8a13-d9bb34b5ca7c admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:32.003292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d20009-4048-4b24-8a13-d9bb34b5ca7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:32.005805 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d20009-4048-4b24-8a13-d9bb34b5ca7c admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:32.016562 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:32.023190 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 914/1853] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:32.024278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d20009-4048-4b24-8a13-d9bb34b5ca7c admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:32.032897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db57a6ac-e0d1-4f02-ad3a-26bdefe1271d admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:32.035899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d20009-4048-4b24-8a13-d9bb34b5ca7c admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:32.039487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87388eb6-183d-4413-9bf3-dc0246b9f306 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:32.040324 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87388eb6-183d-4413-9bf3-dc0246b9f306 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:32.040712 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87388eb6-183d-4413-9bf3-dc0246b9f306 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:32.041362 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87388eb6-183d-4413-9bf3-dc0246b9f306 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:32.042629 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e62575f-a722-433a-8cc8-bd83de4760fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:32.043705 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62575f-a722-433a-8cc8-bd83de4760fe admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:32.043886 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62575f-a722-433a-8cc8-bd83de4760fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:32.044123 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62575f-a722-433a-8cc8-bd83de4760fe admin admin] PATH_INFO: `/v3/projects/3984f694f3fe41bf834deb75418d5230/users/7dfac160848c4fb59454585defeed3d2/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:32.047506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e62575f-a722-433a-8cc8-bd83de4760fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=3984f694f3fe41bf834deb75418d5230, user_id=7dfac160848c4fb59454585defeed3d2, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:32.050498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e62575f-a722-433a-8cc8-bd83de4760fe admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:32.065594 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 915/1854] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:32 2022] PUT /identity/v3/projects/3984f694f3fe41bf834deb75418d5230/users/7dfac160848c4fb59454585defeed3d2/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:32.066546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e62575f-a722-433a-8cc8-bd83de4760fe admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:32.154977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87388eb6-183d-4413-9bf3-dc0246b9f306 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:32.156599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87388eb6-183d-4413-9bf3-dc0246b9f306 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:32.182073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e62575f-a722-433a-8cc8-bd83de4760fe admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:32.188678 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 940/1855] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:32.191567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381ffe9-c600-400c-b7df-f86d6c303e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:32.192642 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381ffe9-c600-400c-b7df-f86d6c303e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:32.193025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381ffe9-c600-400c-b7df-f86d6c303e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:32.193405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381ffe9-c600-400c-b7df-f86d6c303e69 admin admin] PATH_INFO: `/v3/projects/bcdfcfac90ef4eacafb147989e6f62f0/users/272d7c3b843e45fbbd805b345066b953/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:32.198931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1378c5-c5b7-444b-bd8e-951262f51213 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:32.199061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1378c5-c5b7-444b-bd8e-951262f51213 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:32.199277 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1378c5-c5b7-444b-bd8e-951262f51213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:32.210960 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381ffe9-c600-400c-b7df-f86d6c303e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcdfcfac90ef4eacafb147989e6f62f0, user_id=272d7c3b843e45fbbd805b345066b953, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:32.213603 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381ffe9-c600-400c-b7df-f86d6c303e69 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:32.234890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 916/1856] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:32 2022] PUT /identity/v3/projects/bcdfcfac90ef4eacafb147989e6f62f0/users/272d7c3b843e45fbbd805b345066b953/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:32.248147 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdafdeb-bb37-4c9d-af67-8b780dc3a76c None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:32.248448 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-2a1378c5-c5b7-444b-bd8e-951262f51213 None None] MFA Rules not processed for user `7dfac160848c4fb59454585defeed3d2`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:32.248631 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdafdeb-bb37-4c9d-af67-8b780dc3a76c None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:32.249312 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdafdeb-bb37-4c9d-af67-8b780dc3a76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:32.249882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2a1378c5-c5b7-444b-bd8e-951262f51213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:32.295528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-6cdafdeb-bb37-4c9d-af67-8b780dc3a76c None None] MFA Rules not processed for user `272d7c3b843e45fbbd805b345066b953`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:32.303519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-6cdafdeb-bb37-4c9d-af67-8b780dc3a76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:32.399087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 917/1857] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:32.404432 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdafdeb-bb37-4c9d-af67-8b780dc3a76c None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:32.418511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 941/1858] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:32.445710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1378c5-c5b7-444b-bd8e-951262f51213 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:32.447785 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2a1378c5-c5b7-444b-bd8e-951262f51213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:32.554831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:32.579450 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1378c5-c5b7-444b-bd8e-951262f51213 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:32.589157 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a87e3d-a483-463e-84dc-f12db08497fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:32.589913 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a87e3d-a483-463e-84dc-f12db08497fe service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:32.590213 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a87e3d-a483-463e-84dc-f12db08497fe service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:32.590488 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a87e3d-a483-463e-84dc-f12db08497fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:32.590866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdafdeb-bb37-4c9d-af67-8b780dc3a76c None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:32.602052 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cd4da-4a80-4562-9a9f-dcb0e646ce06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:32.604140 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cd4da-4a80-4562-9a9f-dcb0e646ce06 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:32.605242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cd4da-4a80-4562-9a9f-dcb0e646ce06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:32.605918 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cd4da-4a80-4562-9a9f-dcb0e646ce06 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:32.608863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962cd4da-4a80-4562-9a9f-dcb0e646ce06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:32.610982 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a87e3d-a483-463e-84dc-f12db08497fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:32.612791 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962cd4da-4a80-4562-9a9f-dcb0e646ce06 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:32.612923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a87e3d-a483-463e-84dc-f12db08497fe service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:32.633998 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 942/1859] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:48:32.703129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a87e3d-a483-463e-84dc-f12db08497fe service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:32.705810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 918/1860] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:32 2022] POST /identity/v3/projects => generated 383 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:48:32.720564 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a87e3d-a483-463e-84dc-f12db08497fe service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:32.782073 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:32.792562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cd4da-4a80-4562-9a9f-dcb0e646ce06 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:32.887028 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2246b-d234-4825-aaeb-f1c002460cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:32.888009 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2246b-d234-4825-aaeb-f1c002460cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:32.888305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2246b-d234-4825-aaeb-f1c002460cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:32.888566 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2246b-d234-4825-aaeb-f1c002460cc3 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:32.889908 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2246b-d234-4825-aaeb-f1c002460cc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:32.892418 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2246b-d234-4825-aaeb-f1c002460cc3 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:32.920291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cd4da-4a80-4562-9a9f-dcb0e646ce06 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:32.938486 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be0d48-fc47-4951-b53c-48aba5798087 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:32.939514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 943/1861] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:32 2022] POST /identity/v3/users => generated 298 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:48:32.944578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be0d48-fc47-4951-b53c-48aba5798087 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:32.944879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be0d48-fc47-4951-b53c-48aba5798087 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:32.947010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2246b-d234-4825-aaeb-f1c002460cc3 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:32.947447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be0d48-fc47-4951-b53c-48aba5798087 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:32.971147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2246b-d234-4825-aaeb-f1c002460cc3 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:33.001148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c2cd5-6986-48f1-8a99-5a7ebc38650f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:33.003910 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c2cd5-6986-48f1-8a99-5a7ebc38650f admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:33.004225 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c2cd5-6986-48f1-8a99-5a7ebc38650f admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:33.004513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c2cd5-6986-48f1-8a99-5a7ebc38650f admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:33.005420 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c2cd5-6986-48f1-8a99-5a7ebc38650f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:33.016535 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c2cd5-6986-48f1-8a99-5a7ebc38650f admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:33.054437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 944/1862] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:33.059666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c2cd5-6986-48f1-8a99-5a7ebc38650f admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:33.089783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c2cd5-6986-48f1-8a99-5a7ebc38650f admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:33.103708 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ba015-0886-4bca-998b-afeff548e1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:33.104844 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ba015-0886-4bca-998b-afeff548e1ab admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:33.105510 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ba015-0886-4bca-998b-afeff548e1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:33.106129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ba015-0886-4bca-998b-afeff548e1ab admin admin] PATH_INFO: `/v3/projects/d49c4bd82dd84fafbf7c5bf258821ebf/users/2a9732ba10cf40fab9ae5e5dbd0f18d1/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:33.135980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be0d48-fc47-4951-b53c-48aba5798087 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:33.137537 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be0d48-fc47-4951-b53c-48aba5798087 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:33.151167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ba015-0886-4bca-998b-afeff548e1ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=d49c4bd82dd84fafbf7c5bf258821ebf, user_id=2a9732ba10cf40fab9ae5e5dbd0f18d1, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:33.155216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ba015-0886-4bca-998b-afeff548e1ab admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:33.155744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 919/1863] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:48:33.184302 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 945/1864] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:33 2022] PUT /identity/v3/projects/d49c4bd82dd84fafbf7c5bf258821ebf/users/2a9732ba10cf40fab9ae5e5dbd0f18d1/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:33.188276 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be0d48-fc47-4951-b53c-48aba5798087 service cinder] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:33.211394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be0d48-fc47-4951-b53c-48aba5798087 service cinder] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:33.262386 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ba015-0886-4bca-998b-afeff548e1ab admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:33.338496 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851cb2d-d8ce-445b-a663-09777a2e1d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:33.339387 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851cb2d-d8ce-445b-a663-09777a2e1d0c admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:33.339730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851cb2d-d8ce-445b-a663-09777a2e1d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:33.340019 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851cb2d-d8ce-445b-a663-09777a2e1d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:33.340767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851cb2d-d8ce-445b-a663-09777a2e1d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:33.342440 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851cb2d-d8ce-445b-a663-09777a2e1d0c admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:33.361732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 920/1865] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:33.366174 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851cb2d-d8ce-445b-a663-09777a2e1d0c admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:33.390465 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851cb2d-d8ce-445b-a663-09777a2e1d0c admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:33.403122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13052aa-c706-49ac-94be-0585b9ae20e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:33.408344 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13052aa-c706-49ac-94be-0585b9ae20e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:33.408630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13052aa-c706-49ac-94be-0585b9ae20e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:33.408920 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13052aa-c706-49ac-94be-0585b9ae20e8 admin admin] PATH_INFO: `/v3/projects/d49c4bd82dd84fafbf7c5bf258821ebf/users/2a9732ba10cf40fab9ae5e5dbd0f18d1/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:33.414391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13052aa-c706-49ac-94be-0585b9ae20e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d49c4bd82dd84fafbf7c5bf258821ebf, user_id=2a9732ba10cf40fab9ae5e5dbd0f18d1, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:33.417046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ba015-0886-4bca-998b-afeff548e1ab admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:33.419350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13052aa-c706-49ac-94be-0585b9ae20e8 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:33.423003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5bbf6-90c0-4145-8574-5de082e07c5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:33.424274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5bbf6-90c0-4145-8574-5de082e07c5e service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:33.424696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5bbf6-90c0-4145-8574-5de082e07c5e service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:33.424981 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5bbf6-90c0-4145-8574-5de082e07c5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:33.431166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-afc5bbf6-90c0-4145-8574-5de082e07c5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:33.439661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 921/1866] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:33 2022] PUT /identity/v3/projects/d49c4bd82dd84fafbf7c5bf258821ebf/users/2a9732ba10cf40fab9ae5e5dbd0f18d1/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:48:33.454037 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb56f83-a18e-403b-89a2-8453e15d946e None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:33.454344 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb56f83-a18e-403b-89a2-8453e15d946e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:33.454626 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb56f83-a18e-403b-89a2-8453e15d946e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:33.512371 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-0fb56f83-a18e-403b-89a2-8453e15d946e None None] MFA Rules not processed for user `2a9732ba10cf40fab9ae5e5dbd0f18d1`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:33.514729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-0fb56f83-a18e-403b-89a2-8453e15d946e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:33.519271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:33.526997 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5bbf6-90c0-4145-8574-5de082e07c5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:33.528601 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5bbf6-90c0-4145-8574-5de082e07c5e service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:33.538630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 946/1867] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:48:33.593983 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 922/1868] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:33.606083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5bbf6-90c0-4145-8574-5de082e07c5e service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:33.614422 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5bbf6-90c0-4145-8574-5de082e07c5e service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:33.646150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:33.669270 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97341d-ed63-463d-afe6-1a85fa42bbd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:33.669959 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97341d-ed63-463d-afe6-1a85fa42bbd7 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:33.670249 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97341d-ed63-463d-afe6-1a85fa42bbd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:33.670526 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97341d-ed63-463d-afe6-1a85fa42bbd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:33.681852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97341d-ed63-463d-afe6-1a85fa42bbd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:33.683337 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97341d-ed63-463d-afe6-1a85fa42bbd7 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:33.694246 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 947/1869] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:33.806789 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb56f83-a18e-403b-89a2-8453e15d946e None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:33.825406 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb56f83-a18e-403b-89a2-8453e15d946e None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:33.829217 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3df37-5dea-4c1a-8468-54f28719d7ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:33.830159 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3df37-5dea-4c1a-8468-54f28719d7ce service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:33.830906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3df37-5dea-4c1a-8468-54f28719d7ce service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:33.831263 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3df37-5dea-4c1a-8468-54f28719d7ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:33.945969 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3df37-5dea-4c1a-8468-54f28719d7ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:33.947424 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3df37-5dea-4c1a-8468-54f28719d7ce service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:33.957119 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 923/1870] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:39.272710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bff7b-1a09-4361-9f67-ef6d93d4ea35 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:39.273253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bff7b-1a09-4361-9f67-ef6d93d4ea35 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:39.273253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857bff7b-1a09-4361-9f67-ef6d93d4ea35 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:39.275450 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 948/1871] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jun 27 15:48:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:48:39.287233 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1d68e-2a65-437f-98a9-f7686c5e4c77 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:39.287746 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1d68e-2a65-437f-98a9-f7686c5e4c77 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:39.287821 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1d68e-2a65-437f-98a9-f7686c5e4c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:39.327679 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-54e1d68e-2a65-437f-98a9-f7686c5e4c77 None None] MFA Rules not processed for user `1c8f6262cf24419daf803ae71aed24f5`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:39.329167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-54e1d68e-2a65-437f-98a9-f7686c5e4c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:39.428358 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 924/1872] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jun 27 15:48:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:39.443070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bff7b-1a09-4361-9f67-ef6d93d4ea35 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:39.445189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-857bff7b-1a09-4361-9f67-ef6d93d4ea35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:39.468763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857bff7b-1a09-4361-9f67-ef6d93d4ea35 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:39.475809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68b1e8-f156-43e5-9124-f0aa3698d1fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:39.476713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68b1e8-f156-43e5-9124-f0aa3698d1fc service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:39.477151 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68b1e8-f156-43e5-9124-f0aa3698d1fc service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:39.477684 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68b1e8-f156-43e5-9124-f0aa3698d1fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:39.487820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68b1e8-f156-43e5-9124-f0aa3698d1fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:39.489412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68b1e8-f156-43e5-9124-f0aa3698d1fc service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:39.501097 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 949/1873] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:45.429546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e1d68e-2a65-437f-98a9-f7686c5e4c77 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:45.480944 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:45.528816 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e1d68e-2a65-437f-98a9-f7686c5e4c77 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:45.535075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e4a80-b2b5-47fd-8355-9963cef063b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:45.536529 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e4a80-b2b5-47fd-8355-9963cef063b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:45.536815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e4a80-b2b5-47fd-8355-9963cef063b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:45.537214 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e4a80-b2b5-47fd-8355-9963cef063b7 admin admin] PATH_INFO: `/v3/users/272d7c3b843e45fbbd805b345066b953` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:45.540730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e4a80-b2b5-47fd-8355-9963cef063b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=272d7c3b843e45fbbd805b345066b953)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:45.543252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e4a80-b2b5-47fd-8355-9963cef063b7 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:45.626695 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-657e4a80-b2b5-47fd-8355-9963cef063b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '272d7c3b843e45fbbd805b345066b953'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:45.642848 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-657e4a80-b2b5-47fd-8355-9963cef063b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '272d7c3b843e45fbbd805b345066b953'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:45.656331 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-657e4a80-b2b5-47fd-8355-9963cef063b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '272d7c3b843e45fbbd805b345066b953'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:45.657454 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-657e4a80-b2b5-47fd-8355-9963cef063b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '272d7c3b843e45fbbd805b345066b953'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:45.659209 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-657e4a80-b2b5-47fd-8355-9963cef063b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '272d7c3b843e45fbbd805b345066b953'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:45.707697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 925/1874] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:45 2022] DELETE /identity/v3/users/272d7c3b843e45fbbd805b345066b953 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:45.713603 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63557dd4-5116-4b0a-8ed7-8b882c819815 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:45.714070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63557dd4-5116-4b0a-8ed7-8b882c819815 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:45.714274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63557dd4-5116-4b0a-8ed7-8b882c819815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:45.752876 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-63557dd4-5116-4b0a-8ed7-8b882c819815 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:45.756522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-63557dd4-5116-4b0a-8ed7-8b882c819815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:45.856476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 950/1875] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:45.864036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e4a80-b2b5-47fd-8355-9963cef063b7 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:45.866335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-657e4a80-b2b5-47fd-8355-9963cef063b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:45.969348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e4a80-b2b5-47fd-8355-9963cef063b7 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:45.973081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cacfd-f963-4c58-a0f9-d35ea81982a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:45.973814 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cacfd-f963-4c58-a0f9-d35ea81982a2 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:45.974103 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cacfd-f963-4c58-a0f9-d35ea81982a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:45.974382 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cacfd-f963-4c58-a0f9-d35ea81982a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:45.987140 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cacfd-f963-4c58-a0f9-d35ea81982a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:45.989004 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cacfd-f963-4c58-a0f9-d35ea81982a2 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:46.001957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 926/1876] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:46.359155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63557dd4-5116-4b0a-8ed7-8b882c819815 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:46.360635 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-63557dd4-5116-4b0a-8ed7-8b882c819815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:46.375184 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63557dd4-5116-4b0a-8ed7-8b882c819815 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:46.380288 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ffe47-9a20-4edb-8169-c0fe35ca2fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:46.381860 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ffe47-9a20-4edb-8169-c0fe35ca2fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:46.382139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ffe47-9a20-4edb-8169-c0fe35ca2fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:46.382411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ffe47-9a20-4edb-8169-c0fe35ca2fb6 admin admin] PATH_INFO: `/v3/projects/bcdfcfac90ef4eacafb147989e6f62f0` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:46.385012 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ffe47-9a20-4edb-8169-c0fe35ca2fb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcdfcfac90ef4eacafb147989e6f62f0)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:46.386921 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ffe47-9a20-4edb-8169-c0fe35ca2fb6 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:46.436745 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-437ffe47-9a20-4edb-8169-c0fe35ca2fb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcdfcfac90ef4eacafb147989e6f62f0', 'request_id': 'req-437ffe47-9a20-4edb-8169-c0fe35ca2fb6'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:46.442874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-437ffe47-9a20-4edb-8169-c0fe35ca2fb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcdfcfac90ef4eacafb147989e6f62f0', 'request_id': 'req-437ffe47-9a20-4edb-8169-c0fe35ca2fb6'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:46.451353 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-437ffe47-9a20-4edb-8169-c0fe35ca2fb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcdfcfac90ef4eacafb147989e6f62f0', 'request_id': 'req-437ffe47-9a20-4edb-8169-c0fe35ca2fb6'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:46.465750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-437ffe47-9a20-4edb-8169-c0fe35ca2fb6 admin admin] The token cache is being invalidate because project bcdfcfac90ef4eacafb147989e6f62f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:46.466042 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-437ffe47-9a20-4edb-8169-c0fe35ca2fb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:46.466738 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-437ffe47-9a20-4edb-8169-c0fe35ca2fb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:46.468586 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 951/1877] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:46 2022] DELETE /identity/v3/projects/bcdfcfac90ef4eacafb147989e6f62f0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:46.472418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cacfd-f963-4c58-a0f9-d35ea81982a2 service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:46.483585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cacfd-f963-4c58-a0f9-d35ea81982a2 service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:46.538284 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:46.550173 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dcc802-99bd-455e-8441-6a20c31f09e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:46.551058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcc802-99bd-455e-8441-6a20c31f09e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:46.551342 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcc802-99bd-455e-8441-6a20c31f09e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:46.551636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dcc802-99bd-455e-8441-6a20c31f09e5 admin admin] PATH_INFO: `/v3/users/2a9732ba10cf40fab9ae5e5dbd0f18d1` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:46.553183 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dcc802-99bd-455e-8441-6a20c31f09e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a9732ba10cf40fab9ae5e5dbd0f18d1)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:46.555344 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dcc802-99bd-455e-8441-6a20c31f09e5 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:46.604728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-45dcc802-99bd-455e-8441-6a20c31f09e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a9732ba10cf40fab9ae5e5dbd0f18d1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:46.617790 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-45dcc802-99bd-455e-8441-6a20c31f09e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a9732ba10cf40fab9ae5e5dbd0f18d1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:46.631210 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-45dcc802-99bd-455e-8441-6a20c31f09e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a9732ba10cf40fab9ae5e5dbd0f18d1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:46.631210 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-45dcc802-99bd-455e-8441-6a20c31f09e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a9732ba10cf40fab9ae5e5dbd0f18d1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:46.631210 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-45dcc802-99bd-455e-8441-6a20c31f09e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a9732ba10cf40fab9ae5e5dbd0f18d1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:46.643626 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 927/1878] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:46 2022] DELETE /identity/v3/users/2a9732ba10cf40fab9ae5e5dbd0f18d1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:46.929117 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ffe47-9a20-4edb-8169-c0fe35ca2fb6 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:46.930666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-437ffe47-9a20-4edb-8169-c0fe35ca2fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:46.945334 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ffe47-9a20-4edb-8169-c0fe35ca2fb6 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:47.001435 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:47.016071 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74a6b3-45d1-4977-9ad6-f173087e0a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:47.016967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74a6b3-45d1-4977-9ad6-f173087e0a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:47.017254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74a6b3-45d1-4977-9ad6-f173087e0a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:47.017531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74a6b3-45d1-4977-9ad6-f173087e0a85 admin admin] PATH_INFO: `/v3/projects/d49c4bd82dd84fafbf7c5bf258821ebf` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:47.018752 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74a6b3-45d1-4977-9ad6-f173087e0a85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d49c4bd82dd84fafbf7c5bf258821ebf)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:47.020683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74a6b3-45d1-4977-9ad6-f173087e0a85 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:47.073863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-fd74a6b3-45d1-4977-9ad6-f173087e0a85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd49c4bd82dd84fafbf7c5bf258821ebf', 'request_id': 'req-fd74a6b3-45d1-4977-9ad6-f173087e0a85'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:47.080670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-fd74a6b3-45d1-4977-9ad6-f173087e0a85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd49c4bd82dd84fafbf7c5bf258821ebf', 'request_id': 'req-fd74a6b3-45d1-4977-9ad6-f173087e0a85'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:47.091626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-fd74a6b3-45d1-4977-9ad6-f173087e0a85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd49c4bd82dd84fafbf7c5bf258821ebf', 'request_id': 'req-fd74a6b3-45d1-4977-9ad6-f173087e0a85'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:47.107176 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-fd74a6b3-45d1-4977-9ad6-f173087e0a85 admin admin] The token cache is being invalidate because project d49c4bd82dd84fafbf7c5bf258821ebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:47.107532 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-fd74a6b3-45d1-4977-9ad6-f173087e0a85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:47.108143 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-fd74a6b3-45d1-4977-9ad6-f173087e0a85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:47.109971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 952/1879] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:46 2022] DELETE /identity/v3/projects/d49c4bd82dd84fafbf7c5bf258821ebf => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:47.114790 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c8cbf-4bd8-4e00-aca1-e4f17400c225 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:47.115217 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c8cbf-4bd8-4e00-aca1-e4f17400c225 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:47.115636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c8cbf-4bd8-4e00-aca1-e4f17400c225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:47.136556 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-805c8cbf-4bd8-4e00-aca1-e4f17400c225 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:47.137846 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-805c8cbf-4bd8-4e00-aca1-e4f17400c225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:47.212128 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 928/1880] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:48:47.219628 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550177d-2666-49b3-beb1-d61954015257 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:47.219916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550177d-2666-49b3-beb1-d61954015257 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:47.220200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550177d-2666-49b3-beb1-d61954015257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:47.241000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-5550177d-2666-49b3-beb1-d61954015257 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:47.242320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-5550177d-2666-49b3-beb1-d61954015257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:47.250685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 953/1881] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:47.254967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c8cbf-4bd8-4e00-aca1-e4f17400c225 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:47.265375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c8cbf-4bd8-4e00-aca1-e4f17400c225 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:47.270104 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f3230-ceaf-4c03-8f71-2b6a8aac2712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:47.270923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f3230-ceaf-4c03-8f71-2b6a8aac2712 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:47.271220 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f3230-ceaf-4c03-8f71-2b6a8aac2712 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:47.271949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f3230-ceaf-4c03-8f71-2b6a8aac2712 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:47.272384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f3230-ceaf-4c03-8f71-2b6a8aac2712 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:48:47.272701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f3230-ceaf-4c03-8f71-2b6a8aac2712 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:47.274373 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f3230-ceaf-4c03-8f71-2b6a8aac2712 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:47.292274 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 929/1882] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:48:47.296681 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550177d-2666-49b3-beb1-d61954015257 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:47.311050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550177d-2666-49b3-beb1-d61954015257 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:47.316715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ad2a4-8065-4526-96ae-89887cb2ee2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:47.317585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ad2a4-8065-4526-96ae-89887cb2ee2a admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:47.317916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ad2a4-8065-4526-96ae-89887cb2ee2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:47.318221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ad2a4-8065-4526-96ae-89887cb2ee2a admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:47.318932 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ad2a4-8065-4526-96ae-89887cb2ee2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:47.321203 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ad2a4-8065-4526-96ae-89887cb2ee2a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:47.354451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 954/1883] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:47 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:48:47.358501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f3230-ceaf-4c03-8f71-2b6a8aac2712 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:47.368982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f3230-ceaf-4c03-8f71-2b6a8aac2712 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:47.430046 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b559d-2cca-4796-802a-108aa9e362e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:47.430788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b559d-2cca-4796-802a-108aa9e362e7 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:47.431075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b559d-2cca-4796-802a-108aa9e362e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:47.431354 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b559d-2cca-4796-802a-108aa9e362e7 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:47.432061 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b559d-2cca-4796-802a-108aa9e362e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:47.434047 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b559d-2cca-4796-802a-108aa9e362e7 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:47.440962 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:47.461320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 930/1884] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:47 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:47.465614 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ad2a4-8065-4526-96ae-89887cb2ee2a admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:47.477107 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ad2a4-8065-4526-96ae-89887cb2ee2a admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:47.482099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51d7cc-ea30-40e0-8b69-faa13dd71682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:47.482763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51d7cc-ea30-40e0-8b69-faa13dd71682 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:47.483050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51d7cc-ea30-40e0-8b69-faa13dd71682 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:47.483331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51d7cc-ea30-40e0-8b69-faa13dd71682 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:47.484061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51d7cc-ea30-40e0-8b69-faa13dd71682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:47.485683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51d7cc-ea30-40e0-8b69-faa13dd71682 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:47.496859 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 955/1885] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:47.500765 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b559d-2cca-4796-802a-108aa9e362e7 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:47.510834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b559d-2cca-4796-802a-108aa9e362e7 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:47.515433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ae750-68de-4006-801e-ba1002271ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:47.516091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ae750-68de-4006-801e-ba1002271ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:47.516376 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ae750-68de-4006-801e-ba1002271ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:47.516653 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ae750-68de-4006-801e-ba1002271ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:47.517382 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ae750-68de-4006-801e-ba1002271ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:47.519025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ae750-68de-4006-801e-ba1002271ae1 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:47.529143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 931/1886] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:47.533337 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51d7cc-ea30-40e0-8b69-faa13dd71682 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:47.544497 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51d7cc-ea30-40e0-8b69-faa13dd71682 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:47.549516 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a1f36-ba7f-4975-9366-05674c6666be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:47.550344 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a1f36-ba7f-4975-9366-05674c6666be admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:47.550632 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a1f36-ba7f-4975-9366-05674c6666be admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:47.550912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a1f36-ba7f-4975-9366-05674c6666be admin admin] PATH_INFO: `/v3/projects/d56f9960829f4a9ba98efbc1df445ba9/users/27a6d4bc25134e939cb62b4284af7d82/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:47.566363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a1f36-ba7f-4975-9366-05674c6666be admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56f9960829f4a9ba98efbc1df445ba9, user_id=27a6d4bc25134e939cb62b4284af7d82, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:47.569424 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a1f36-ba7f-4975-9366-05674c6666be admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:47.582673 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 956/1887] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:47 2022] PUT /identity/v3/projects/d56f9960829f4a9ba98efbc1df445ba9/users/27a6d4bc25134e939cb62b4284af7d82/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:47.593681 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66899a5-7313-47c9-9d14-0232c32f33af None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:47.593975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66899a5-7313-47c9-9d14-0232c32f33af None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:47.594247 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66899a5-7313-47c9-9d14-0232c32f33af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:47.622806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-e66899a5-7313-47c9-9d14-0232c32f33af None None] MFA Rules not processed for user `27a6d4bc25134e939cb62b4284af7d82`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:47.624047 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e66899a5-7313-47c9-9d14-0232c32f33af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:47.689833 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 932/1888] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:47.694558 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a1f36-ba7f-4975-9366-05674c6666be admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:47.792731 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a1f36-ba7f-4975-9366-05674c6666be admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:47.806919 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d28d7-ca27-4969-8ad2-2607230cead2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:47.807654 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d28d7-ca27-4969-8ad2-2607230cead2 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:47.807948 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d28d7-ca27-4969-8ad2-2607230cead2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:47.808227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d28d7-ca27-4969-8ad2-2607230cead2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:47.809024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d28d7-ca27-4969-8ad2-2607230cead2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:47.811235 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d28d7-ca27-4969-8ad2-2607230cead2 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:47.834977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:47.842752 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 957/1889] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:47 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:47.846612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66899a5-7313-47c9-9d14-0232c32f33af None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:47.857345 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66899a5-7313-47c9-9d14-0232c32f33af None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:47.914454 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea81894-6c48-4cff-af55-bdb5b379a133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:47.915126 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea81894-6c48-4cff-af55-bdb5b379a133 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:47.915407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea81894-6c48-4cff-af55-bdb5b379a133 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:47.915681 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea81894-6c48-4cff-af55-bdb5b379a133 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:47.916374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea81894-6c48-4cff-af55-bdb5b379a133 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:47.918320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea81894-6c48-4cff-af55-bdb5b379a133 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:47.938120 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 933/1890] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:47 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:47.942067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d28d7-ca27-4969-8ad2-2607230cead2 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:47.952427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d28d7-ca27-4969-8ad2-2607230cead2 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:47.957028 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670fa1f-8041-469c-80ab-3eea29747f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:47.957725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670fa1f-8041-469c-80ab-3eea29747f30 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:47.958012 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670fa1f-8041-469c-80ab-3eea29747f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:47.958293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670fa1f-8041-469c-80ab-3eea29747f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:47.959585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2670fa1f-8041-469c-80ab-3eea29747f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:47.961105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2670fa1f-8041-469c-80ab-3eea29747f30 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:47.971580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 958/1891] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:47.976087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea81894-6c48-4cff-af55-bdb5b379a133 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:47.988366 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea81894-6c48-4cff-af55-bdb5b379a133 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:47.994883 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bf83b-629f-40ea-8434-74e337b743a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:47.995834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bf83b-629f-40ea-8434-74e337b743a2 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:47.996062 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bf83b-629f-40ea-8434-74e337b743a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:47.996580 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bf83b-629f-40ea-8434-74e337b743a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:48.007497 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9bf83b-629f-40ea-8434-74e337b743a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:48.011251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9bf83b-629f-40ea-8434-74e337b743a2 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:48.032754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 934/1892] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:48:48.036805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670fa1f-8041-469c-80ab-3eea29747f30 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:48.047557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670fa1f-8041-469c-80ab-3eea29747f30 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:48.052357 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5950cb15-191b-4e13-b09c-d0b9b9fd3b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:48.053675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950cb15-191b-4e13-b09c-d0b9b9fd3b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:48.053784 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950cb15-191b-4e13-b09c-d0b9b9fd3b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:48.054002 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950cb15-191b-4e13-b09c-d0b9b9fd3b25 admin admin] PATH_INFO: `/v3/projects/15e21cb4daf54d65baeb30dc8b7db59a/users/4b7afaf8b78648b0bed7831dec8ed839/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:48.070402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5950cb15-191b-4e13-b09c-d0b9b9fd3b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e21cb4daf54d65baeb30dc8b7db59a, user_id=4b7afaf8b78648b0bed7831dec8ed839, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:48.073042 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5950cb15-191b-4e13-b09c-d0b9b9fd3b25 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:48.086912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 959/1893] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:48 2022] PUT /identity/v3/projects/15e21cb4daf54d65baeb30dc8b7db59a/users/4b7afaf8b78648b0bed7831dec8ed839/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:48.096922 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c297d-a353-4617-8c2c-58bab073790c None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:48.097301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c297d-a353-4617-8c2c-58bab073790c None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:48.097764 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c297d-a353-4617-8c2c-58bab073790c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:48.126898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-7b5c297d-a353-4617-8c2c-58bab073790c None None] MFA Rules not processed for user `4b7afaf8b78648b0bed7831dec8ed839`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:48.128176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-7b5c297d-a353-4617-8c2c-58bab073790c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:48.171437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:48.211779 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 935/1894] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:48.216215 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5950cb15-191b-4e13-b09c-d0b9b9fd3b25 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:48.280678 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5950cb15-191b-4e13-b09c-d0b9b9fd3b25 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:48.285607 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b8cae-c5e7-4eb3-b8ff-13b01e5809aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:48.286320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b8cae-c5e7-4eb3-b8ff-13b01e5809aa admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:48.286610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b8cae-c5e7-4eb3-b8ff-13b01e5809aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:48.287605 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b8cae-c5e7-4eb3-b8ff-13b01e5809aa admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:48.288321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b8cae-c5e7-4eb3-b8ff-13b01e5809aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:48.290274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b8cae-c5e7-4eb3-b8ff-13b01e5809aa admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:48.319048 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 960/1895] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:48 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:48.322207 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c297d-a353-4617-8c2c-58bab073790c None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:48.334045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c297d-a353-4617-8c2c-58bab073790c None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:48.416521 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8817b-9c8f-40de-98c3-f796a5bb5827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:48.417295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8817b-9c8f-40de-98c3-f796a5bb5827 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:48.417748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8817b-9c8f-40de-98c3-f796a5bb5827 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:48.418130 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8817b-9c8f-40de-98c3-f796a5bb5827 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:48.418896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8817b-9c8f-40de-98c3-f796a5bb5827 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:48.420885 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8817b-9c8f-40de-98c3-f796a5bb5827 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:48.444527 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 936/1896] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:48 2022] POST /identity/v3/users => generated 296 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:48.448675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b8cae-c5e7-4eb3-b8ff-13b01e5809aa admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:48.462214 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b8cae-c5e7-4eb3-b8ff-13b01e5809aa admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:48.467296 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d11c7f-7f39-42b0-bf09-79b2d22f149e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:48.468099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d11c7f-7f39-42b0-bf09-79b2d22f149e admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:48.468456 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d11c7f-7f39-42b0-bf09-79b2d22f149e admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:48.468805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d11c7f-7f39-42b0-bf09-79b2d22f149e admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:48.469608 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d11c7f-7f39-42b0-bf09-79b2d22f149e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:48.471308 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d11c7f-7f39-42b0-bf09-79b2d22f149e admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:48.484134 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 961/1897] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:48.488305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8817b-9c8f-40de-98c3-f796a5bb5827 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:48.499063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8817b-9c8f-40de-98c3-f796a5bb5827 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:48.504838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010bc19-832f-4cdc-b564-d62e772df4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:48.505852 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010bc19-832f-4cdc-b564-d62e772df4b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:48.506137 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010bc19-832f-4cdc-b564-d62e772df4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:48.506410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010bc19-832f-4cdc-b564-d62e772df4b3 admin admin] PATH_INFO: `/v3/projects/3152e3d88f0a4ca991a2f24f5c196316/users/fd4feec23318495eac196076fc983b92/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:48.523153 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6010bc19-832f-4cdc-b564-d62e772df4b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3152e3d88f0a4ca991a2f24f5c196316, user_id=fd4feec23318495eac196076fc983b92, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:48.525888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6010bc19-832f-4cdc-b564-d62e772df4b3 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:48.541592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 937/1898] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:48 2022] PUT /identity/v3/projects/3152e3d88f0a4ca991a2f24f5c196316/users/fd4feec23318495eac196076fc983b92/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:48.545347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d11c7f-7f39-42b0-bf09-79b2d22f149e admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:48.557349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d11c7f-7f39-42b0-bf09-79b2d22f149e admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:48.618742 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c962a-7ba8-44f2-a500-3bfd24068e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:48.619414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c962a-7ba8-44f2-a500-3bfd24068e89 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:48.619696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c962a-7ba8-44f2-a500-3bfd24068e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:48.619977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c962a-7ba8-44f2-a500-3bfd24068e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:48.620807 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c962a-7ba8-44f2-a500-3bfd24068e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:48.622543 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c962a-7ba8-44f2-a500-3bfd24068e89 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:48.634018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 962/1899] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:48:48.638390 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010bc19-832f-4cdc-b564-d62e772df4b3 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:48.649138 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010bc19-832f-4cdc-b564-d62e772df4b3 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:48.653950 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22880796-71d5-48f7-a249-e526683e6c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:48.654843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22880796-71d5-48f7-a249-e526683e6c79 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:48.655126 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22880796-71d5-48f7-a249-e526683e6c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:48.655402 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22880796-71d5-48f7-a249-e526683e6c79 admin admin] PATH_INFO: `/v3/projects/3152e3d88f0a4ca991a2f24f5c196316/users/fd4feec23318495eac196076fc983b92/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:48.657369 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22880796-71d5-48f7-a249-e526683e6c79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3152e3d88f0a4ca991a2f24f5c196316, user_id=fd4feec23318495eac196076fc983b92, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:48.660068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22880796-71d5-48f7-a249-e526683e6c79 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:48.671720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 938/1900] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:48 2022] PUT /identity/v3/projects/3152e3d88f0a4ca991a2f24f5c196316/users/fd4feec23318495eac196076fc983b92/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:48:48.684047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5651e8-40fd-4ce6-9e45-6a9cdf127411 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:48.684344 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5651e8-40fd-4ce6-9e45-6a9cdf127411 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:48.684623 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5651e8-40fd-4ce6-9e45-6a9cdf127411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:48.711403 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-ae5651e8-40fd-4ce6-9e45-6a9cdf127411 None None] MFA Rules not processed for user `fd4feec23318495eac196076fc983b92`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:48.712685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-ae5651e8-40fd-4ce6-9e45-6a9cdf127411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:48.735738 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:48.786599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 963/1901] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:48.799091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22880796-71d5-48f7-a249-e526683e6c79 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:48.800771 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-22880796-71d5-48f7-a249-e526683e6c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:48.862857 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22880796-71d5-48f7-a249-e526683e6c79 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:48.867336 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4feb0a-5f5d-4c4f-97f6-8542718a2164 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:48.867996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4feb0a-5f5d-4c4f-97f6-8542718a2164 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:48.868282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4feb0a-5f5d-4c4f-97f6-8542718a2164 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:48.868562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4feb0a-5f5d-4c4f-97f6-8542718a2164 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:48.877951 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4feb0a-5f5d-4c4f-97f6-8542718a2164 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:48.879383 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4feb0a-5f5d-4c4f-97f6-8542718a2164 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:48.889050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 939/1902] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:48.929285 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5651e8-40fd-4ce6-9e45-6a9cdf127411 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:48.930698 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-ae5651e8-40fd-4ce6-9e45-6a9cdf127411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:48.942455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5651e8-40fd-4ce6-9e45-6a9cdf127411 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:48.946442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9ccfd-0cd7-44d3-848f-7225ef34a004 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:48.947091 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9ccfd-0cd7-44d3-848f-7225ef34a004 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:48.947374 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9ccfd-0cd7-44d3-848f-7225ef34a004 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:48.947648 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9ccfd-0cd7-44d3-848f-7225ef34a004 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:49.005354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9ccfd-0cd7-44d3-848f-7225ef34a004 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:49.007229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9ccfd-0cd7-44d3-848f-7225ef34a004 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:49.016230 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 964/1903] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:48:49.637457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733777ce-d2a6-4a87-bc07-dd3fcdfd46ea None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:49.638145 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733777ce-d2a6-4a87-bc07-dd3fcdfd46ea None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:49.638145 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733777ce-d2a6-4a87-bc07-dd3fcdfd46ea None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:49.640758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 940/1904] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jun 27 15:48:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:48:49.649702 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c49489-a1ae-4e33-82f1-0474ab5d9622 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:49.650052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c49489-a1ae-4e33-82f1-0474ab5d9622 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:49.650411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c49489-a1ae-4e33-82f1-0474ab5d9622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:49.672324 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-f7c49489-a1ae-4e33-82f1-0474ab5d9622 None None] MFA Rules not processed for user `1c8f6262cf24419daf803ae71aed24f5`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:49.673663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f7c49489-a1ae-4e33-82f1-0474ab5d9622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:49.756071 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 965/1905] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jun 27 15:48:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:49.827622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733777ce-d2a6-4a87-bc07-dd3fcdfd46ea None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:49.829540 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-733777ce-d2a6-4a87-bc07-dd3fcdfd46ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:49.847791 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733777ce-d2a6-4a87-bc07-dd3fcdfd46ea None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:49.855551 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb74ab-d456-4639-8195-e41c3edfddb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:49.856218 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb74ab-d456-4639-8195-e41c3edfddb5 service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:49.856503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb74ab-d456-4639-8195-e41c3edfddb5 service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:49.856786 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb74ab-d456-4639-8195-e41c3edfddb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:49.867667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb74ab-d456-4639-8195-e41c3edfddb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:49.869323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb74ab-d456-4639-8195-e41c3edfddb5 service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:49.885388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 941/1906] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:50.769319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c49489-a1ae-4e33-82f1-0474ab5d9622 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:50.780968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c49489-a1ae-4e33-82f1-0474ab5d9622 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:50.784868 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2b84c-242f-4b0c-b0eb-d60841ba5c04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:50.785517 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2b84c-242f-4b0c-b0eb-d60841ba5c04 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:50.785836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2b84c-242f-4b0c-b0eb-d60841ba5c04 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:50.786111 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2b84c-242f-4b0c-b0eb-d60841ba5c04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:50.788051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-02f2b84c-242f-4b0c-b0eb-d60841ba5c04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:50.865874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:50.889834 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2b84c-242f-4b0c-b0eb-d60841ba5c04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:50.891834 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2b84c-242f-4b0c-b0eb-d60841ba5c04 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:50.904078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 966/1907] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:51.217452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb74ab-d456-4639-8195-e41c3edfddb5 service nova] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:51.219462 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d6cb74ab-d456-4639-8195-e41c3edfddb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:51.250812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:51.340081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb74ab-d456-4639-8195-e41c3edfddb5 service nova] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:51.345740 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6973b-9bc2-463a-8679-2620748d98a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:51.346820 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6973b-9bc2-463a-8679-2620748d98a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:51.347278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6973b-9bc2-463a-8679-2620748d98a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:51.348088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6973b-9bc2-463a-8679-2620748d98a6 admin admin] PATH_INFO: `/v3/users/df380f002e294de3b8aaaa80f6b8e9a3` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:51.350772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e6973b-9bc2-463a-8679-2620748d98a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df380f002e294de3b8aaaa80f6b8e9a3)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:51.352734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e6973b-9bc2-463a-8679-2620748d98a6 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:51.416241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b5e6973b-9bc2-463a-8679-2620748d98a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df380f002e294de3b8aaaa80f6b8e9a3'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:51.442547 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b5e6973b-9bc2-463a-8679-2620748d98a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df380f002e294de3b8aaaa80f6b8e9a3'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:51.452941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b5e6973b-9bc2-463a-8679-2620748d98a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df380f002e294de3b8aaaa80f6b8e9a3'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:51.454914 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b5e6973b-9bc2-463a-8679-2620748d98a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df380f002e294de3b8aaaa80f6b8e9a3'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:51.455990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b5e6973b-9bc2-463a-8679-2620748d98a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df380f002e294de3b8aaaa80f6b8e9a3'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:51.480932 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 942/1908] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:51 2022] DELETE /identity/v3/users/df380f002e294de3b8aaaa80f6b8e9a3 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:51.482913 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4c47f-92b3-4e9c-a863-b279bde9df4b None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:51.483396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4c47f-92b3-4e9c-a863-b279bde9df4b None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:51.483680 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4c47f-92b3-4e9c-a863-b279bde9df4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:51.511234 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-30d4c47f-92b3-4e9c-a863-b279bde9df4b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:51.512783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-30d4c47f-92b3-4e9c-a863-b279bde9df4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:51.622466 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 967/1909] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:51.632934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6973b-9bc2-463a-8679-2620748d98a6 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:51.634637 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-b5e6973b-9bc2-463a-8679-2620748d98a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:51.739482 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6973b-9bc2-463a-8679-2620748d98a6 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:51.746879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a9198-a8c0-4fd7-95d8-0fcbba1b26e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:51.749974 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a9198-a8c0-4fd7-95d8-0fcbba1b26e6 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:51.750361 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a9198-a8c0-4fd7-95d8-0fcbba1b26e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:51.750696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a9198-a8c0-4fd7-95d8-0fcbba1b26e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:51.775221 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a9198-a8c0-4fd7-95d8-0fcbba1b26e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:51.776750 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a9198-a8c0-4fd7-95d8-0fcbba1b26e6 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:51.797824 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 943/1910] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:52.134904 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4c47f-92b3-4e9c-a863-b279bde9df4b None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:52.136390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-30d4c47f-92b3-4e9c-a863-b279bde9df4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:52.149476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4c47f-92b3-4e9c-a863-b279bde9df4b None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:52.155489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0c286-ecab-4055-bb07-00400dd61310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:52.156658 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0c286-ecab-4055-bb07-00400dd61310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:52.157010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0c286-ecab-4055-bb07-00400dd61310 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:52.157360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0c286-ecab-4055-bb07-00400dd61310 admin admin] PATH_INFO: `/v3/users/27a6d4bc25134e939cb62b4284af7d82` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:52.158717 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0c286-ecab-4055-bb07-00400dd61310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27a6d4bc25134e939cb62b4284af7d82)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:52.160721 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0c286-ecab-4055-bb07-00400dd61310 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:52.230671 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a9198-a8c0-4fd7-95d8-0fcbba1b26e6 service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:52.231174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-60e0c286-ecab-4055-bb07-00400dd61310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27a6d4bc25134e939cb62b4284af7d82'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:52.236090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-747a9198-a8c0-4fd7-95d8-0fcbba1b26e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:52.253849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-60e0c286-ecab-4055-bb07-00400dd61310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27a6d4bc25134e939cb62b4284af7d82'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:52.257984 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-60e0c286-ecab-4055-bb07-00400dd61310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27a6d4bc25134e939cb62b4284af7d82'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:52.258510 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a9198-a8c0-4fd7-95d8-0fcbba1b26e6 service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:52.268519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eca30d-49fd-4b63-975c-351b8f13f1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:52.268750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-60e0c286-ecab-4055-bb07-00400dd61310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27a6d4bc25134e939cb62b4284af7d82'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:52.270952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eca30d-49fd-4b63-975c-351b8f13f1e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:52.271996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eca30d-49fd-4b63-975c-351b8f13f1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:52.272935 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eca30d-49fd-4b63-975c-351b8f13f1e2 admin admin] PATH_INFO: `/v3/projects/a06d17b0a34443889304dd82a1148821` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:52.275792 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eca30d-49fd-4b63-975c-351b8f13f1e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a06d17b0a34443889304dd82a1148821)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:52.279857 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-60e0c286-ecab-4055-bb07-00400dd61310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27a6d4bc25134e939cb62b4284af7d82'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:52.280569 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eca30d-49fd-4b63-975c-351b8f13f1e2 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:52.290992 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 968/1911] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:52 2022] DELETE /identity/v3/users/27a6d4bc25134e939cb62b4284af7d82 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:52.295305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc51520-044d-4c70-a868-dd60b4199a30 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:52.295793 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc51520-044d-4c70-a868-dd60b4199a30 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:52.296569 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc51520-044d-4c70-a868-dd60b4199a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:52.316160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-4dc51520-044d-4c70-a868-dd60b4199a30 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:52.317361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-4dc51520-044d-4c70-a868-dd60b4199a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:52.338942 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:52.357582 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-64eca30d-49fd-4b63-975c-351b8f13f1e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a06d17b0a34443889304dd82a1148821', 'request_id': 'req-64eca30d-49fd-4b63-975c-351b8f13f1e2'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:52.364753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-64eca30d-49fd-4b63-975c-351b8f13f1e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a06d17b0a34443889304dd82a1148821', 'request_id': 'req-64eca30d-49fd-4b63-975c-351b8f13f1e2'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:52.371455 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-64eca30d-49fd-4b63-975c-351b8f13f1e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a06d17b0a34443889304dd82a1148821', 'request_id': 'req-64eca30d-49fd-4b63-975c-351b8f13f1e2'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:52.389747 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-64eca30d-49fd-4b63-975c-351b8f13f1e2 admin admin] The token cache is being invalidate because project a06d17b0a34443889304dd82a1148821 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:52.390313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-64eca30d-49fd-4b63-975c-351b8f13f1e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:52.390836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:52.391244 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-64eca30d-49fd-4b63-975c-351b8f13f1e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:52.393325 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 944/1912] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:52 2022] DELETE /identity/v3/projects/a06d17b0a34443889304dd82a1148821 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:52.398712 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eca30d-49fd-4b63-975c-351b8f13f1e2 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:52.400492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-64eca30d-49fd-4b63-975c-351b8f13f1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:52.414111 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 969/1913] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:48:52.422094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc51520-044d-4c70-a868-dd60b4199a30 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:52.423505 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-4dc51520-044d-4c70-a868-dd60b4199a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:52.486912 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eca30d-49fd-4b63-975c-351b8f13f1e2 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:52.492964 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233fad5-9b48-48de-ac97-a8d991e2e78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:52.494247 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233fad5-9b48-48de-ac97-a8d991e2e78b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:52.494633 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233fad5-9b48-48de-ac97-a8d991e2e78b admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:52.495017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233fad5-9b48-48de-ac97-a8d991e2e78b admin admin] PATH_INFO: `/v3/users/7dfac160848c4fb59454585defeed3d2` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:52.496524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233fad5-9b48-48de-ac97-a8d991e2e78b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dfac160848c4fb59454585defeed3d2)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:52.498674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233fad5-9b48-48de-ac97-a8d991e2e78b admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:52.507498 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc51520-044d-4c70-a868-dd60b4199a30 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:52.563636 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e52b8-f843-46e5-a3d1-1874d6796828 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:52.564352 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e52b8-f843-46e5-a3d1-1874d6796828 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:52.564642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e52b8-f843-46e5-a3d1-1874d6796828 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:52.564924 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e52b8-f843-46e5-a3d1-1874d6796828 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:52.566068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a233fad5-9b48-48de-ac97-a8d991e2e78b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dfac160848c4fb59454585defeed3d2'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:52.576552 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e52b8-f843-46e5-a3d1-1874d6796828 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:52.578347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e52b8-f843-46e5-a3d1-1874d6796828 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:52.580631 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a233fad5-9b48-48de-ac97-a8d991e2e78b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dfac160848c4fb59454585defeed3d2'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:52.591824 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a233fad5-9b48-48de-ac97-a8d991e2e78b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dfac160848c4fb59454585defeed3d2'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:52.592513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a233fad5-9b48-48de-ac97-a8d991e2e78b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dfac160848c4fb59454585defeed3d2'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:52.593173 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 970/1914] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:48:52.593308 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a233fad5-9b48-48de-ac97-a8d991e2e78b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dfac160848c4fb59454585defeed3d2'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:52.608005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 945/1915] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:52 2022] DELETE /identity/v3/users/7dfac160848c4fb59454585defeed3d2 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:52.891822 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e52b8-f843-46e5-a3d1-1874d6796828 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:52.893262 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-9a7e52b8-f843-46e5-a3d1-1874d6796828 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:52.909972 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233fad5-9b48-48de-ac97-a8d991e2e78b admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:52.911620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a233fad5-9b48-48de-ac97-a8d991e2e78b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:52.922366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e52b8-f843-46e5-a3d1-1874d6796828 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:52.974887 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:53.007909 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e8fa9-5ed9-4c74-a248-5e0cf2f3c9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:53.008892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e8fa9-5ed9-4c74-a248-5e0cf2f3c9bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:53.009195 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e8fa9-5ed9-4c74-a248-5e0cf2f3c9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:53.009472 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e8fa9-5ed9-4c74-a248-5e0cf2f3c9bb admin admin] PATH_INFO: `/v3/projects/d56f9960829f4a9ba98efbc1df445ba9` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:53.010301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233fad5-9b48-48de-ac97-a8d991e2e78b admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:53.010752 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e8fa9-5ed9-4c74-a248-5e0cf2f3c9bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d56f9960829f4a9ba98efbc1df445ba9)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:53.012835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e8fa9-5ed9-4c74-a248-5e0cf2f3c9bb admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:53.015276 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0a2e0-8b86-4c09-9d6b-10992d5e6b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:53.016356 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0a2e0-8b86-4c09-9d6b-10992d5e6b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:53.016714 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0a2e0-8b86-4c09-9d6b-10992d5e6b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:53.017056 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0a2e0-8b86-4c09-9d6b-10992d5e6b0e admin admin] PATH_INFO: `/v3/projects/3984f694f3fe41bf834deb75418d5230` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:53.018357 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0a2e0-8b86-4c09-9d6b-10992d5e6b0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3984f694f3fe41bf834deb75418d5230)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:53.020335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0a2e0-8b86-4c09-9d6b-10992d5e6b0e admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:53.077982 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-f40e8fa9-5ed9-4c74-a248-5e0cf2f3c9bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd56f9960829f4a9ba98efbc1df445ba9', 'request_id': 'req-f40e8fa9-5ed9-4c74-a248-5e0cf2f3c9bb'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.079050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-1ec0a2e0-8b86-4c09-9d6b-10992d5e6b0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3984f694f3fe41bf834deb75418d5230', 'request_id': 'req-1ec0a2e0-8b86-4c09-9d6b-10992d5e6b0e'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.085292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-f40e8fa9-5ed9-4c74-a248-5e0cf2f3c9bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd56f9960829f4a9ba98efbc1df445ba9', 'request_id': 'req-f40e8fa9-5ed9-4c74-a248-5e0cf2f3c9bb'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.090308 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-1ec0a2e0-8b86-4c09-9d6b-10992d5e6b0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3984f694f3fe41bf834deb75418d5230', 'request_id': 'req-1ec0a2e0-8b86-4c09-9d6b-10992d5e6b0e'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.095907 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-f40e8fa9-5ed9-4c74-a248-5e0cf2f3c9bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd56f9960829f4a9ba98efbc1df445ba9', 'request_id': 'req-f40e8fa9-5ed9-4c74-a248-5e0cf2f3c9bb'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.097460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:53.097607 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-1ec0a2e0-8b86-4c09-9d6b-10992d5e6b0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3984f694f3fe41bf834deb75418d5230', 'request_id': 'req-1ec0a2e0-8b86-4c09-9d6b-10992d5e6b0e'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.117394 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-f40e8fa9-5ed9-4c74-a248-5e0cf2f3c9bb admin admin] The token cache is being invalidate because project d56f9960829f4a9ba98efbc1df445ba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:53.118065 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-f40e8fa9-5ed9-4c74-a248-5e0cf2f3c9bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.118832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-1ec0a2e0-8b86-4c09-9d6b-10992d5e6b0e admin admin] The token cache is being invalidate because project 3984f694f3fe41bf834deb75418d5230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:53.119209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-f40e8fa9-5ed9-4c74-a248-5e0cf2f3c9bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.120203 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-1ec0a2e0-8b86-4c09-9d6b-10992d5e6b0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.121606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-1ec0a2e0-8b86-4c09-9d6b-10992d5e6b0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.121903 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 971/1916] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:52 2022] DELETE /identity/v3/projects/d56f9960829f4a9ba98efbc1df445ba9 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:53.123559 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 946/1917] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:52 2022] DELETE /identity/v3/projects/3984f694f3fe41bf834deb75418d5230 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:48:53.129723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e8fa9-5ed9-4c74-a248-5e0cf2f3c9bb admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:53.131242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77695dd3-3fde-4878-8b84-4e7fb3073809 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:53.131831 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f40e8fa9-5ed9-4c74-a248-5e0cf2f3c9bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:53.132427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77695dd3-3fde-4878-8b84-4e7fb3073809 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:53.132867 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77695dd3-3fde-4878-8b84-4e7fb3073809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:53.157908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-77695dd3-3fde-4878-8b84-4e7fb3073809 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:53.159105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-77695dd3-3fde-4878-8b84-4e7fb3073809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:53.202338 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e8fa9-5ed9-4c74-a248-5e0cf2f3c9bb admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:53.222843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 947/1918] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:53.230391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1db97-5ff5-4b1b-84c6-95063d48e074 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:53.230707 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1db97-5ff5-4b1b-84c6-95063d48e074 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:53.231251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1db97-5ff5-4b1b-84c6-95063d48e074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:53.249880 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-79c1db97-5ff5-4b1b-84c6-95063d48e074 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:53.251094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-79c1db97-5ff5-4b1b-84c6-95063d48e074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:53.260240 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 948/1919] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:48:53.264293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1db97-5ff5-4b1b-84c6-95063d48e074 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:53.265559 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa49952-600f-4cdf-8502-d5921902b470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:53.266466 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa49952-600f-4cdf-8502-d5921902b470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:53.266747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa49952-600f-4cdf-8502-d5921902b470 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:53.267020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa49952-600f-4cdf-8502-d5921902b470 admin admin] PATH_INFO: `/v3/users/4b7afaf8b78648b0bed7831dec8ed839` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:53.268223 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa49952-600f-4cdf-8502-d5921902b470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b7afaf8b78648b0bed7831dec8ed839)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:53.270179 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa49952-600f-4cdf-8502-d5921902b470 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:53.274213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1db97-5ff5-4b1b-84c6-95063d48e074 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:53.278894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e37b3f-e767-4250-8109-648852167fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:53.279540 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e37b3f-e767-4250-8109-648852167fbd admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:53.279823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e37b3f-e767-4250-8109-648852167fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:53.280156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e37b3f-e767-4250-8109-648852167fbd admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:53.280821 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e37b3f-e767-4250-8109-648852167fbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:48:53.281124 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e37b3f-e767-4250-8109-648852167fbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:53.282741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e37b3f-e767-4250-8109-648852167fbd admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:53.294721 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 949/1920] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:48:53.299605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e37b3f-e767-4250-8109-648852167fbd admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:53.310307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e37b3f-e767-4250-8109-648852167fbd admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:53.315994 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb8e52-2a79-4662-a1af-ac3fd7e1290e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:53.316654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb8e52-2a79-4662-a1af-ac3fd7e1290e admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:53.317063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb8e52-2a79-4662-a1af-ac3fd7e1290e admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:53.317346 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb8e52-2a79-4662-a1af-ac3fd7e1290e admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:53.318662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb8e52-2a79-4662-a1af-ac3fd7e1290e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:53.320808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fb8e52-2a79-4662-a1af-ac3fd7e1290e admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:53.322866 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-aaa49952-600f-4cdf-8502-d5921902b470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b7afaf8b78648b0bed7831dec8ed839'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.337336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-aaa49952-600f-4cdf-8502-d5921902b470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b7afaf8b78648b0bed7831dec8ed839'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.338216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-aaa49952-600f-4cdf-8502-d5921902b470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b7afaf8b78648b0bed7831dec8ed839'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.346375 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-aaa49952-600f-4cdf-8502-d5921902b470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b7afaf8b78648b0bed7831dec8ed839'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.349130 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 950/1921] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:53 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:48:53.353844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb8e52-2a79-4662-a1af-ac3fd7e1290e admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:53.356721 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e5fb8e52-2a79-4662-a1af-ac3fd7e1290e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:53.358274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-aaa49952-600f-4cdf-8502-d5921902b470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b7afaf8b78648b0bed7831dec8ed839'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.365964 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 972/1922] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:53 2022] DELETE /identity/v3/users/4b7afaf8b78648b0bed7831dec8ed839 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:53.397458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:53.440596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb8e52-2a79-4662-a1af-ac3fd7e1290e admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:53.514301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8f1b3-4fb4-4a63-b6f3-7417c012ef13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:53.514983 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8f1b3-4fb4-4a63-b6f3-7417c012ef13 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:53.515265 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8f1b3-4fb4-4a63-b6f3-7417c012ef13 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:53.515544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8f1b3-4fb4-4a63-b6f3-7417c012ef13 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:53.516229 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8f1b3-4fb4-4a63-b6f3-7417c012ef13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:53.523541 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8f1b3-4fb4-4a63-b6f3-7417c012ef13 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:53.541580 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 951/1923] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:53 2022] POST /identity/v3/users => generated 282 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:53.547245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa49952-600f-4cdf-8502-d5921902b470 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:53.558966 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa49952-600f-4cdf-8502-d5921902b470 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:53.563579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301f8ef-fee4-4980-bdff-6c8b6923b026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:53.564280 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301f8ef-fee4-4980-bdff-6c8b6923b026 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:53.564564 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301f8ef-fee4-4980-bdff-6c8b6923b026 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:53.564842 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301f8ef-fee4-4980-bdff-6c8b6923b026 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:53.565572 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301f8ef-fee4-4980-bdff-6c8b6923b026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:53.567199 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301f8ef-fee4-4980-bdff-6c8b6923b026 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:53.578318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 973/1924] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:48:53.582450 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8f1b3-4fb4-4a63-b6f3-7417c012ef13 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:53.593024 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8f1b3-4fb4-4a63-b6f3-7417c012ef13 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:53.598405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a86818-6a0f-45d6-a94e-77aeb2677389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:53.599165 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a86818-6a0f-45d6-a94e-77aeb2677389 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:53.600131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a86818-6a0f-45d6-a94e-77aeb2677389 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:53.600471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a86818-6a0f-45d6-a94e-77aeb2677389 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:53.601343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a86818-6a0f-45d6-a94e-77aeb2677389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:53.602994 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a86818-6a0f-45d6-a94e-77aeb2677389 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:53.619637 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 952/1925] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:53.620880 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301f8ef-fee4-4980-bdff-6c8b6923b026 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:53.629597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:53.633803 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301f8ef-fee4-4980-bdff-6c8b6923b026 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:53.645388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585fbb9-b7f8-453d-9019-30479ba0ff64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:53.646972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585fbb9-b7f8-453d-9019-30479ba0ff64 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:53.647350 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585fbb9-b7f8-453d-9019-30479ba0ff64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:53.647660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585fbb9-b7f8-453d-9019-30479ba0ff64 admin admin] PATH_INFO: `/v3/projects/ffbdc6bee1c744918f6d6aa74d45304b/users/20d89b1e5f194b3b9081b474f3ecb51a/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:53.664735 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585fbb9-b7f8-453d-9019-30479ba0ff64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffbdc6bee1c744918f6d6aa74d45304b, user_id=20d89b1e5f194b3b9081b474f3ecb51a, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:53.667501 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585fbb9-b7f8-453d-9019-30479ba0ff64 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:53.684391 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 974/1926] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:53 2022] PUT /identity/v3/projects/ffbdc6bee1c744918f6d6aa74d45304b/users/20d89b1e5f194b3b9081b474f3ecb51a/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:53.695273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31be005-a86f-4abd-a99d-3fd0b866ac0c None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:53.695585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31be005-a86f-4abd-a99d-3fd0b866ac0c None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:53.695873 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31be005-a86f-4abd-a99d-3fd0b866ac0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:53.756690 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-c31be005-a86f-4abd-a99d-3fd0b866ac0c None None] MFA Rules not processed for user `20d89b1e5f194b3b9081b474f3ecb51a`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:53.758550 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-c31be005-a86f-4abd-a99d-3fd0b866ac0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:53.778698 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585fbb9-b7f8-453d-9019-30479ba0ff64 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:53.780526 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-9585fbb9-b7f8-453d-9019-30479ba0ff64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:53.867356 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 953/1927] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:53 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:53.871561 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31be005-a86f-4abd-a99d-3fd0b866ac0c None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:53.878282 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585fbb9-b7f8-453d-9019-30479ba0ff64 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:53.883055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31be005-a86f-4abd-a99d-3fd0b866ac0c None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:53.883962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e332a04-d94c-41cb-b393-22e21e45a83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:53.884931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e332a04-d94c-41cb-b393-22e21e45a83c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:53.885411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e332a04-d94c-41cb-b393-22e21e45a83c admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:53.885815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e332a04-d94c-41cb-b393-22e21e45a83c admin admin] PATH_INFO: `/v3/projects/15e21cb4daf54d65baeb30dc8b7db59a` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:53.887196 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e332a04-d94c-41cb-b393-22e21e45a83c admin admin] RBAC: Authorizing `identity:delete_project(project_id=15e21cb4daf54d65baeb30dc8b7db59a)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:53.888002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb389ebb-4c74-44c4-b36c-42c430214249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:53.888682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb389ebb-4c74-44c4-b36c-42c430214249 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:53.888972 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb389ebb-4c74-44c4-b36c-42c430214249 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:53.889456 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e332a04-d94c-41cb-b393-22e21e45a83c admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:53.889571 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb389ebb-4c74-44c4-b36c-42c430214249 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:53.890264 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb389ebb-4c74-44c4-b36c-42c430214249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:53.892771 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb389ebb-4c74-44c4-b36c-42c430214249 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:53.925415 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 954/1928] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:53 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:48:53.926385 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb389ebb-4c74-44c4-b36c-42c430214249 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:53.944728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-3e332a04-d94c-41cb-b393-22e21e45a83c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15e21cb4daf54d65baeb30dc8b7db59a', 'request_id': 'req-3e332a04-d94c-41cb-b393-22e21e45a83c'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.950968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-3e332a04-d94c-41cb-b393-22e21e45a83c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15e21cb4daf54d65baeb30dc8b7db59a', 'request_id': 'req-3e332a04-d94c-41cb-b393-22e21e45a83c'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.954395 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:53.959487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-3e332a04-d94c-41cb-b393-22e21e45a83c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15e21cb4daf54d65baeb30dc8b7db59a', 'request_id': 'req-3e332a04-d94c-41cb-b393-22e21e45a83c'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.970861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-3e332a04-d94c-41cb-b393-22e21e45a83c admin admin] The token cache is being invalidate because project 15e21cb4daf54d65baeb30dc8b7db59a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:53.971180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-3e332a04-d94c-41cb-b393-22e21e45a83c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.971852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-3e332a04-d94c-41cb-b393-22e21e45a83c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:53.973626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 975/1929] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:53 2022] DELETE /identity/v3/projects/15e21cb4daf54d65baeb30dc8b7db59a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:53.978007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e332a04-d94c-41cb-b393-22e21e45a83c admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:53.979331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-3e332a04-d94c-41cb-b393-22e21e45a83c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:53.994430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb389ebb-4c74-44c4-b36c-42c430214249 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:54.047616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e332a04-d94c-41cb-b393-22e21e45a83c admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:54.051305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f22115-89de-4c32-a093-d1f48543b4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:54.052013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f22115-89de-4c32-a093-d1f48543b4f1 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:54.052336 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f22115-89de-4c32-a093-d1f48543b4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:54.052442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d277dd4-326f-40f2-b996-5f24ed9b6bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:54.052768 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f22115-89de-4c32-a093-d1f48543b4f1 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:54.053222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d277dd4-326f-40f2-b996-5f24ed9b6bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:54.053481 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f22115-89de-4c32-a093-d1f48543b4f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:54.053585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d277dd4-326f-40f2-b996-5f24ed9b6bce admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:54.053808 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d277dd4-326f-40f2-b996-5f24ed9b6bce admin admin] PATH_INFO: `/v3/users/fd4feec23318495eac196076fc983b92` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:54.055058 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d277dd4-326f-40f2-b996-5f24ed9b6bce admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd4feec23318495eac196076fc983b92)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:54.055436 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f22115-89de-4c32-a093-d1f48543b4f1 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:54.057004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d277dd4-326f-40f2-b996-5f24ed9b6bce admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:54.071068 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:54.084379 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 955/1930] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:53 2022] POST /identity/v3/users => generated 288 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:48:54.088499 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f22115-89de-4c32-a093-d1f48543b4f1 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:54.090361 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-77f22115-89de-4c32-a093-d1f48543b4f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:54.106883 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f22115-89de-4c32-a093-d1f48543b4f1 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:54.114298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f332f-a11d-4011-bd26-c015ba208e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:54.115102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f332f-a11d-4011-bd26-c015ba208e29 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:54.115284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-3d277dd4-326f-40f2-b996-5f24ed9b6bce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd4feec23318495eac196076fc983b92'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:54.115676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f332f-a11d-4011-bd26-c015ba208e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:54.116022 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f332f-a11d-4011-bd26-c015ba208e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:54.116842 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f332f-a11d-4011-bd26-c015ba208e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:54.118602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f332f-a11d-4011-bd26-c015ba208e29 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:54.130209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-3d277dd4-326f-40f2-b996-5f24ed9b6bce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd4feec23318495eac196076fc983b92'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:54.131762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-3d277dd4-326f-40f2-b996-5f24ed9b6bce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd4feec23318495eac196076fc983b92'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:54.132899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 956/1931] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:48:54.137003 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f332f-a11d-4011-bd26-c015ba208e29 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:54.138988 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-7d9f332f-a11d-4011-bd26-c015ba208e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:54.140677 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-3d277dd4-326f-40f2-b996-5f24ed9b6bce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd4feec23318495eac196076fc983b92'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:54.148698 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-3d277dd4-326f-40f2-b996-5f24ed9b6bce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd4feec23318495eac196076fc983b92'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:54.153489 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f332f-a11d-4011-bd26-c015ba208e29 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:54.155063 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 976/1932] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:53 2022] DELETE /identity/v3/users/fd4feec23318495eac196076fc983b92 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:48:54.228922 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8714853-dcfa-4a49-b428-e1ac616d20c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:54.229959 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8714853-dcfa-4a49-b428-e1ac616d20c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:54.230240 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8714853-dcfa-4a49-b428-e1ac616d20c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:54.230517 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8714853-dcfa-4a49-b428-e1ac616d20c1 admin admin] PATH_INFO: `/v3/projects/a36d24d8aaca4f9a86e7dd3d3a7bef2c/users/07fbbdbf625b492784c3d57090daa50d/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:54.247585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8714853-dcfa-4a49-b428-e1ac616d20c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a36d24d8aaca4f9a86e7dd3d3a7bef2c, user_id=07fbbdbf625b492784c3d57090daa50d, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:54.250339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8714853-dcfa-4a49-b428-e1ac616d20c1 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:54.265002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 957/1933] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:54 2022] PUT /identity/v3/projects/a36d24d8aaca4f9a86e7dd3d3a7bef2c/users/07fbbdbf625b492784c3d57090daa50d/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:48:54.269030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d277dd4-326f-40f2-b996-5f24ed9b6bce admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:54.281256 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d277dd4-326f-40f2-b996-5f24ed9b6bce admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:54.349900 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13f5ee-f6e8-4f4e-9568-40c66721f3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:54.351568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13f5ee-f6e8-4f4e-9568-40c66721f3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:54.351861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13f5ee-f6e8-4f4e-9568-40c66721f3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:54.352137 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13f5ee-f6e8-4f4e-9568-40c66721f3e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:54.352931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13f5ee-f6e8-4f4e-9568-40c66721f3e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:54.354543 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13f5ee-f6e8-4f4e-9568-40c66721f3e0 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:54.366780 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 977/1934] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:54.370806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8714853-dcfa-4a49-b428-e1ac616d20c1 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:54.382425 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8714853-dcfa-4a49-b428-e1ac616d20c1 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:54.387160 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e665d7-9d46-4c86-9be1-c4d5241132fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:54.388092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e665d7-9d46-4c86-9be1-c4d5241132fd admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:54.388373 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e665d7-9d46-4c86-9be1-c4d5241132fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:54.388650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e665d7-9d46-4c86-9be1-c4d5241132fd admin admin] PATH_INFO: `/v3/projects/a36d24d8aaca4f9a86e7dd3d3a7bef2c/users/07fbbdbf625b492784c3d57090daa50d/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:54.390911 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e665d7-9d46-4c86-9be1-c4d5241132fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a36d24d8aaca4f9a86e7dd3d3a7bef2c, user_id=07fbbdbf625b492784c3d57090daa50d, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:54.393590 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e665d7-9d46-4c86-9be1-c4d5241132fd admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:54.405978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 958/1935] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:54 2022] PUT /identity/v3/projects/a36d24d8aaca4f9a86e7dd3d3a7bef2c/users/07fbbdbf625b492784c3d57090daa50d/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:54.417965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922de4e6-2f5e-4967-8f25-d982678028eb None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:54.418261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922de4e6-2f5e-4967-8f25-d982678028eb None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:54.418543 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922de4e6-2f5e-4967-8f25-d982678028eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:54.449427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e665d7-9d46-4c86-9be1-c4d5241132fd admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:54.451199 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f8e665d7-9d46-4c86-9be1-c4d5241132fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:54.453085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-922de4e6-2f5e-4967-8f25-d982678028eb None None] MFA Rules not processed for user `07fbbdbf625b492784c3d57090daa50d`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:54.454772 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-922de4e6-2f5e-4967-8f25-d982678028eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:54.492629 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:54.546129 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e665d7-9d46-4c86-9be1-c4d5241132fd admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:54.549835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 978/1936] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:54 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:48:54.553458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f6e6b-c333-4542-9c4b-5f3905f44446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:54.554469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f6e6b-c333-4542-9c4b-5f3905f44446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:54.554750 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f6e6b-c333-4542-9c4b-5f3905f44446 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:54.555031 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f6e6b-c333-4542-9c4b-5f3905f44446 admin admin] PATH_INFO: `/v3/projects/3152e3d88f0a4ca991a2f24f5c196316` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:54.556301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f6e6b-c333-4542-9c4b-5f3905f44446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3152e3d88f0a4ca991a2f24f5c196316)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:54.558274 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f6e6b-c333-4542-9c4b-5f3905f44446 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:54.559715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922de4e6-2f5e-4967-8f25-d982678028eb None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:54.561335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-922de4e6-2f5e-4967-8f25-d982678028eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:54.577546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:54.616303 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-305f6e6b-c333-4542-9c4b-5f3905f44446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3152e3d88f0a4ca991a2f24f5c196316', 'request_id': 'req-305f6e6b-c333-4542-9c4b-5f3905f44446'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:54.626047 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-305f6e6b-c333-4542-9c4b-5f3905f44446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3152e3d88f0a4ca991a2f24f5c196316', 'request_id': 'req-305f6e6b-c333-4542-9c4b-5f3905f44446'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:54.634220 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-305f6e6b-c333-4542-9c4b-5f3905f44446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3152e3d88f0a4ca991a2f24f5c196316', 'request_id': 'req-305f6e6b-c333-4542-9c4b-5f3905f44446'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:54.637327 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922de4e6-2f5e-4967-8f25-d982678028eb None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:54.650456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-305f6e6b-c333-4542-9c4b-5f3905f44446 admin admin] The token cache is being invalidate because project 3152e3d88f0a4ca991a2f24f5c196316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:54.650755 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-305f6e6b-c333-4542-9c4b-5f3905f44446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:54.651446 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-305f6e6b-c333-4542-9c4b-5f3905f44446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:54.653791 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 959/1937] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:54 2022] DELETE /identity/v3/projects/3152e3d88f0a4ca991a2f24f5c196316 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:48:54.659424 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195be72-e217-44f6-be37-60c075ce534c None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:54.659798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195be72-e217-44f6-be37-60c075ce534c None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:54.660208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195be72-e217-44f6-be37-60c075ce534c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:54.681507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-1195be72-e217-44f6-be37-60c075ce534c None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:54.682849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1195be72-e217-44f6-be37-60c075ce534c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:54.695292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb240dd-1a7a-4a00-b595-d12b35f3f678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:54.695889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb240dd-1a7a-4a00-b595-d12b35f3f678 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:54.696170 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb240dd-1a7a-4a00-b595-d12b35f3f678 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:54.696457 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb240dd-1a7a-4a00-b595-d12b35f3f678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:54.698309 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-4bb240dd-1a7a-4a00-b595-d12b35f3f678 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:54.775692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 960/1938] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:48:54.783969 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce5040-a041-45d4-96c7-b8a5a3c966ed None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:54.784264 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce5040-a041-45d4-96c7-b8a5a3c966ed None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:54.784733 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce5040-a041-45d4-96c7-b8a5a3c966ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:54.811136 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb240dd-1a7a-4a00-b595-d12b35f3f678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:54.812948 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb240dd-1a7a-4a00-b595-d12b35f3f678 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:54.814663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-0bce5040-a041-45d4-96c7-b8a5a3c966ed None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:54.816016 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-0bce5040-a041-45d4-96c7-b8a5a3c966ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:54.830527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 979/1939] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:54 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:48:54.849905 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 961/1940] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:54.851368 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb240dd-1a7a-4a00-b595-d12b35f3f678 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:54.867695 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb240dd-1a7a-4a00-b595-d12b35f3f678 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:54.873178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192bae1-a587-4153-a3cf-2b6da770c4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:54.873962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192bae1-a587-4153-a3cf-2b6da770c4ea admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:54.874238 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192bae1-a587-4153-a3cf-2b6da770c4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:54.874523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192bae1-a587-4153-a3cf-2b6da770c4ea admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:54.877964 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192bae1-a587-4153-a3cf-2b6da770c4ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:48:54.878710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192bae1-a587-4153-a3cf-2b6da770c4ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:54.882776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192bae1-a587-4153-a3cf-2b6da770c4ea admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:54.903360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 980/1941] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:48:54.907930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce5040-a041-45d4-96c7-b8a5a3c966ed None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:54.924367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce5040-a041-45d4-96c7-b8a5a3c966ed None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:54.928410 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192bae1-a587-4153-a3cf-2b6da770c4ea admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:54.930648 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e192bae1-a587-4153-a3cf-2b6da770c4ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:54.931125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ca0f4-3f13-4d01-8a16-7998485cb34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:54.932269 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ca0f4-3f13-4d01-8a16-7998485cb34b admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:54.932696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ca0f4-3f13-4d01-8a16-7998485cb34b admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:54.933159 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ca0f4-3f13-4d01-8a16-7998485cb34b admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:54.934084 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ca0f4-3f13-4d01-8a16-7998485cb34b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:54.936300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ca0f4-3f13-4d01-8a16-7998485cb34b admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:54.948675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192bae1-a587-4153-a3cf-2b6da770c4ea admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:54.954303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1132295-a799-4cd9-8a00-c38243211d37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:54.955341 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1132295-a799-4cd9-8a00-c38243211d37 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:54.955819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1132295-a799-4cd9-8a00-c38243211d37 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:54.956537 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1132295-a799-4cd9-8a00-c38243211d37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:54.959312 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-a1132295-a799-4cd9-8a00-c38243211d37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:54.980000 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 962/1942] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:54 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:54.984960 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ca0f4-3f13-4d01-8a16-7998485cb34b admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:55.042723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:55.087327 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1132295-a799-4cd9-8a00-c38243211d37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:55.090517 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1132295-a799-4cd9-8a00-c38243211d37 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:55.105815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ca0f4-3f13-4d01-8a16-7998485cb34b admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:55.112465 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 981/1943] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:54 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:48:55.143335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ecc43-6f1f-474d-8b64-c05055dd5763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:55.145248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ecc43-6f1f-474d-8b64-c05055dd5763 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:55.145248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ecc43-6f1f-474d-8b64-c05055dd5763 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:55.145248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ecc43-6f1f-474d-8b64-c05055dd5763 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:55.148333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40ecc43-6f1f-474d-8b64-c05055dd5763 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:55.152765 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40ecc43-6f1f-474d-8b64-c05055dd5763 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:55.221182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 963/1944] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:54 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:55.223492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1132295-a799-4cd9-8a00-c38243211d37 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:55.232180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:55.242842 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1132295-a799-4cd9-8a00-c38243211d37 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:55.274542 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e014da-22b2-4990-90ae-8b3bf7f00d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:55.278435 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e014da-22b2-4990-90ae-8b3bf7f00d25 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:55.278845 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e014da-22b2-4990-90ae-8b3bf7f00d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:55.279239 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e014da-22b2-4990-90ae-8b3bf7f00d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:55.282200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e014da-22b2-4990-90ae-8b3bf7f00d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:55.285775 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e014da-22b2-4990-90ae-8b3bf7f00d25 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:55.316393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 982/1945] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:48:55.321066 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ecc43-6f1f-474d-8b64-c05055dd5763 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:55.335617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ecc43-6f1f-474d-8b64-c05055dd5763 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:55.360062 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f7df5-f133-4d1c-ac44-bc5310596fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:55.361032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f7df5-f133-4d1c-ac44-bc5310596fab admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:55.361731 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f7df5-f133-4d1c-ac44-bc5310596fab admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:55.362493 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f7df5-f133-4d1c-ac44-bc5310596fab admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:55.365274 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f7df5-f133-4d1c-ac44-bc5310596fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:55.367029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f7df5-f133-4d1c-ac44-bc5310596fab admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:55.393372 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 964/1946] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:48:55.394610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e014da-22b2-4990-90ae-8b3bf7f00d25 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:55.408382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e014da-22b2-4990-90ae-8b3bf7f00d25 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:55.427168 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ebf7f-9eac-41e7-a82d-f754d8586291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:55.428886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ebf7f-9eac-41e7-a82d-f754d8586291 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:55.429175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ebf7f-9eac-41e7-a82d-f754d8586291 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:55.430133 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ebf7f-9eac-41e7-a82d-f754d8586291 admin admin] PATH_INFO: `/v3/projects/77a747d7554f4c07b9a1bc41b45af171/users/4aa2f60039e84581b7f9715a68e691a1/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:55.453836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ebf7f-9eac-41e7-a82d-f754d8586291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77a747d7554f4c07b9a1bc41b45af171, user_id=4aa2f60039e84581b7f9715a68e691a1, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:55.456954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ebf7f-9eac-41e7-a82d-f754d8586291 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:55.484586 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 983/1947] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:55 2022] PUT /identity/v3/projects/77a747d7554f4c07b9a1bc41b45af171/users/4aa2f60039e84581b7f9715a68e691a1/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:48:55.496045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2f3d0-54d3-41f5-a733-838a386d35fe None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:55.496456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2f3d0-54d3-41f5-a733-838a386d35fe None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:55.496930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2f3d0-54d3-41f5-a733-838a386d35fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:55.549107 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-e1f2f3d0-54d3-41f5-a733-838a386d35fe None None] MFA Rules not processed for user `4aa2f60039e84581b7f9715a68e691a1`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:55.550876 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e1f2f3d0-54d3-41f5-a733-838a386d35fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:55.694599 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 965/1948] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:55.733980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ebf7f-9eac-41e7-a82d-f754d8586291 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:55.735759 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-7e3ebf7f-9eac-41e7-a82d-f754d8586291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:55.797306 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ebf7f-9eac-41e7-a82d-f754d8586291 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:55.801519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7532e5-ff40-4907-b0da-cf6f9b381b50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:55.802208 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7532e5-ff40-4907-b0da-cf6f9b381b50 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:55.803069 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7532e5-ff40-4907-b0da-cf6f9b381b50 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:55.803345 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7532e5-ff40-4907-b0da-cf6f9b381b50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:55.812557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7532e5-ff40-4907-b0da-cf6f9b381b50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:55.814021 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7532e5-ff40-4907-b0da-cf6f9b381b50 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:55.826546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 984/1949] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:55.964360 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2f3d0-54d3-41f5-a733-838a386d35fe None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:55.965819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e1f2f3d0-54d3-41f5-a733-838a386d35fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:56.057133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2f3d0-54d3-41f5-a733-838a386d35fe None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:56.061951 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4f8a9-7629-478e-8b3d-9f8c1e8f1116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:56.062825 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4f8a9-7629-478e-8b3d-9f8c1e8f1116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:56.063113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4f8a9-7629-478e-8b3d-9f8c1e8f1116 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:56.063388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4f8a9-7629-478e-8b3d-9f8c1e8f1116 admin admin] PATH_INFO: `/v3/users/83936890f89f4073999f1643874c3c0c` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:56.064761 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4f8a9-7629-478e-8b3d-9f8c1e8f1116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83936890f89f4073999f1643874c3c0c)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:56.067000 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4f8a9-7629-478e-8b3d-9f8c1e8f1116 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:56.117333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:56.146060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3fd4f8a9-7629-478e-8b3d-9f8c1e8f1116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83936890f89f4073999f1643874c3c0c'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:56.166245 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3fd4f8a9-7629-478e-8b3d-9f8c1e8f1116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83936890f89f4073999f1643874c3c0c'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:56.188182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3fd4f8a9-7629-478e-8b3d-9f8c1e8f1116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83936890f89f4073999f1643874c3c0c'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:56.189123 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3fd4f8a9-7629-478e-8b3d-9f8c1e8f1116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83936890f89f4073999f1643874c3c0c'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:56.189897 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3fd4f8a9-7629-478e-8b3d-9f8c1e8f1116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83936890f89f4073999f1643874c3c0c'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:56.226499 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 966/1950] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:55 2022] DELETE /identity/v3/users/83936890f89f4073999f1643874c3c0c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:56.230914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348159d2-c9cf-45ca-ae34-27c892ecca37 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:56.231394 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348159d2-c9cf-45ca-ae34-27c892ecca37 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:56.231877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348159d2-c9cf-45ca-ae34-27c892ecca37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:56.260428 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-348159d2-c9cf-45ca-ae34-27c892ecca37 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:56.264489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-348159d2-c9cf-45ca-ae34-27c892ecca37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:56.350948 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 985/1951] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:56.360872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4f8a9-7629-478e-8b3d-9f8c1e8f1116 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:56.362404 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3fd4f8a9-7629-478e-8b3d-9f8c1e8f1116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:56.448147 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4f8a9-7629-478e-8b3d-9f8c1e8f1116 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:56.452081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b2e2e-9c95-4ba3-9d1d-8788860afaf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:56.452850 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b2e2e-9c95-4ba3-9d1d-8788860afaf9 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:56.453057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b2e2e-9c95-4ba3-9d1d-8788860afaf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:56.453347 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b2e2e-9c95-4ba3-9d1d-8788860afaf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:56.464409 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b2e2e-9c95-4ba3-9d1d-8788860afaf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:56.471179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b2e2e-9c95-4ba3-9d1d-8788860afaf9 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:56.479098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 967/1952] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:48:56.824288 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348159d2-c9cf-45ca-ae34-27c892ecca37 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:56.825818 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-348159d2-c9cf-45ca-ae34-27c892ecca37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:56.856174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348159d2-c9cf-45ca-ae34-27c892ecca37 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:56.862258 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b425a-d21b-432e-beee-506fdc297079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:56.863321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b425a-d21b-432e-beee-506fdc297079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:56.863668 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b425a-d21b-432e-beee-506fdc297079 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:56.864009 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b425a-d21b-432e-beee-506fdc297079 admin admin] PATH_INFO: `/v3/projects/1fa9bbae31c44721acbbb57f6d2a3de0` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:56.867331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b425a-d21b-432e-beee-506fdc297079 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fa9bbae31c44721acbbb57f6d2a3de0)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:56.870213 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b425a-d21b-432e-beee-506fdc297079 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:56.958836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-617b425a-d21b-432e-beee-506fdc297079 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fa9bbae31c44721acbbb57f6d2a3de0', 'request_id': 'req-617b425a-d21b-432e-beee-506fdc297079'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:56.965230 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-617b425a-d21b-432e-beee-506fdc297079 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fa9bbae31c44721acbbb57f6d2a3de0', 'request_id': 'req-617b425a-d21b-432e-beee-506fdc297079'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:56.972987 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-617b425a-d21b-432e-beee-506fdc297079 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fa9bbae31c44721acbbb57f6d2a3de0', 'request_id': 'req-617b425a-d21b-432e-beee-506fdc297079'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:56.978034 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:56.986268 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-617b425a-d21b-432e-beee-506fdc297079 admin admin] The token cache is being invalidate because project 1fa9bbae31c44721acbbb57f6d2a3de0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:56.986901 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-617b425a-d21b-432e-beee-506fdc297079 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:56.987682 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-617b425a-d21b-432e-beee-506fdc297079 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:56.989483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 986/1953] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:56 2022] DELETE /identity/v3/projects/1fa9bbae31c44721acbbb57f6d2a3de0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:56.993945 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b2e2e-9c95-4ba3-9d1d-8788860afaf9 service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:57.004569 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b2e2e-9c95-4ba3-9d1d-8788860afaf9 service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:57.068862 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff7bab-c5ff-46c1-847e-e70ec370fbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:57.069775 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff7bab-c5ff-46c1-847e-e70ec370fbc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:57.070058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff7bab-c5ff-46c1-847e-e70ec370fbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:57.070333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff7bab-c5ff-46c1-847e-e70ec370fbc4 admin admin] PATH_INFO: `/v3/users/c071987e60d4429dbfb2c9a587461044` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:57.071534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff7bab-c5ff-46c1-847e-e70ec370fbc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c071987e60d4429dbfb2c9a587461044)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:57.073458 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff7bab-c5ff-46c1-847e-e70ec370fbc4 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:57.122425 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e1ff7bab-c5ff-46c1-847e-e70ec370fbc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c071987e60d4429dbfb2c9a587461044'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:57.128643 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:57.137762 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e1ff7bab-c5ff-46c1-847e-e70ec370fbc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c071987e60d4429dbfb2c9a587461044'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:57.148118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e1ff7bab-c5ff-46c1-847e-e70ec370fbc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c071987e60d4429dbfb2c9a587461044'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:57.148816 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e1ff7bab-c5ff-46c1-847e-e70ec370fbc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c071987e60d4429dbfb2c9a587461044'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:57.149446 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e1ff7bab-c5ff-46c1-847e-e70ec370fbc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c071987e60d4429dbfb2c9a587461044'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:57.163398 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 968/1954] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:56 2022] DELETE /identity/v3/users/c071987e60d4429dbfb2c9a587461044 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:57.444226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b425a-d21b-432e-beee-506fdc297079 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:57.445935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-617b425a-d21b-432e-beee-506fdc297079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:57.465714 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b425a-d21b-432e-beee-506fdc297079 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:57.531530 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49423375-db73-4736-bcdc-c47d41173850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:57.532416 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49423375-db73-4736-bcdc-c47d41173850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:57.532700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49423375-db73-4736-bcdc-c47d41173850 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:57.533009 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49423375-db73-4736-bcdc-c47d41173850 admin admin] PATH_INFO: `/v3/projects/0b7483726a8546188eb3b7ae221ca8da` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:57.534243 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49423375-db73-4736-bcdc-c47d41173850 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b7483726a8546188eb3b7ae221ca8da)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:57.536468 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49423375-db73-4736-bcdc-c47d41173850 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:57.588947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-49423375-db73-4736-bcdc-c47d41173850 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b7483726a8546188eb3b7ae221ca8da', 'request_id': 'req-49423375-db73-4736-bcdc-c47d41173850'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:57.598339 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-49423375-db73-4736-bcdc-c47d41173850 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b7483726a8546188eb3b7ae221ca8da', 'request_id': 'req-49423375-db73-4736-bcdc-c47d41173850'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:57.604528 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-49423375-db73-4736-bcdc-c47d41173850 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b7483726a8546188eb3b7ae221ca8da', 'request_id': 'req-49423375-db73-4736-bcdc-c47d41173850'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:57.615106 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-49423375-db73-4736-bcdc-c47d41173850 admin admin] The token cache is being invalidate because project 0b7483726a8546188eb3b7ae221ca8da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:57.615375 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-49423375-db73-4736-bcdc-c47d41173850 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:57.616050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-49423375-db73-4736-bcdc-c47d41173850 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:57.617570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 987/1955] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:57 2022] DELETE /identity/v3/projects/0b7483726a8546188eb3b7ae221ca8da => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:48:57.621542 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff7bab-c5ff-46c1-847e-e70ec370fbc4 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:57.622990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e1ff7bab-c5ff-46c1-847e-e70ec370fbc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:57.635633 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff7bab-c5ff-46c1-847e-e70ec370fbc4 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:57.693243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596cda9f-040f-4516-a2e0-69ddea0a9a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:57.694197 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cda9f-040f-4516-a2e0-69ddea0a9a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:57.694482 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cda9f-040f-4516-a2e0-69ddea0a9a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:57.694760 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596cda9f-040f-4516-a2e0-69ddea0a9a29 admin admin] PATH_INFO: `/v3/users/2adde4db87ba473a9dfb5b2078f18eb5` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:57.695998 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596cda9f-040f-4516-a2e0-69ddea0a9a29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2adde4db87ba473a9dfb5b2078f18eb5)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:57.697993 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596cda9f-040f-4516-a2e0-69ddea0a9a29 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:57.749851 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-596cda9f-040f-4516-a2e0-69ddea0a9a29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2adde4db87ba473a9dfb5b2078f18eb5'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:57.764439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-596cda9f-040f-4516-a2e0-69ddea0a9a29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2adde4db87ba473a9dfb5b2078f18eb5'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:57.786598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-596cda9f-040f-4516-a2e0-69ddea0a9a29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2adde4db87ba473a9dfb5b2078f18eb5'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:57.789746 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-596cda9f-040f-4516-a2e0-69ddea0a9a29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2adde4db87ba473a9dfb5b2078f18eb5'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:57.790427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-596cda9f-040f-4516-a2e0-69ddea0a9a29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2adde4db87ba473a9dfb5b2078f18eb5'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:57.805374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 969/1956] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:57 2022] DELETE /identity/v3/users/2adde4db87ba473a9dfb5b2078f18eb5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:58.175530 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49423375-db73-4736-bcdc-c47d41173850 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:58.178437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-49423375-db73-4736-bcdc-c47d41173850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:58.201464 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49423375-db73-4736-bcdc-c47d41173850 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:58.262628 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee9aa4-4cc0-4728-b16c-b3867bb90419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:58.263898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee9aa4-4cc0-4728-b16c-b3867bb90419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:58.264261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee9aa4-4cc0-4728-b16c-b3867bb90419 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:58.264578 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee9aa4-4cc0-4728-b16c-b3867bb90419 admin admin] PATH_INFO: `/v3/projects/1e4ae28c23cf4df19940018b5e6d4a6b` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:58.265850 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee9aa4-4cc0-4728-b16c-b3867bb90419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e4ae28c23cf4df19940018b5e6d4a6b)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:58.267782 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee9aa4-4cc0-4728-b16c-b3867bb90419 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:58.310517 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:58.330098 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-6eee9aa4-4cc0-4728-b16c-b3867bb90419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e4ae28c23cf4df19940018b5e6d4a6b', 'request_id': 'req-6eee9aa4-4cc0-4728-b16c-b3867bb90419'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:58.337200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-6eee9aa4-4cc0-4728-b16c-b3867bb90419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e4ae28c23cf4df19940018b5e6d4a6b', 'request_id': 'req-6eee9aa4-4cc0-4728-b16c-b3867bb90419'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:58.345801 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-6eee9aa4-4cc0-4728-b16c-b3867bb90419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e4ae28c23cf4df19940018b5e6d4a6b', 'request_id': 'req-6eee9aa4-4cc0-4728-b16c-b3867bb90419'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:58.357317 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-6eee9aa4-4cc0-4728-b16c-b3867bb90419 admin admin] The token cache is being invalidate because project 1e4ae28c23cf4df19940018b5e6d4a6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:48:58.357591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-6eee9aa4-4cc0-4728-b16c-b3867bb90419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:58.358293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-6eee9aa4-4cc0-4728-b16c-b3867bb90419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:58.364428 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 988/1957] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:48:58 2022] DELETE /identity/v3/projects/1e4ae28c23cf4df19940018b5e6d4a6b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:58.366819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111d3a3-d96c-4d40-9427-9f2895b6f31d None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:58.367117 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111d3a3-d96c-4d40-9427-9f2895b6f31d None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:58.367394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111d3a3-d96c-4d40-9427-9f2895b6f31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:58.386744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-3111d3a3-d96c-4d40-9427-9f2895b6f31d None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:58.387974 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3111d3a3-d96c-4d40-9427-9f2895b6f31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:58.435091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:58.451429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 970/1958] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:48:58.461445 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab63064-bac3-42cd-8a21-c17581035614 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:58.461995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab63064-bac3-42cd-8a21-c17581035614 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:58.462080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab63064-bac3-42cd-8a21-c17581035614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:58.482524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-0ab63064-bac3-42cd-8a21-c17581035614 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:58.483982 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0ab63064-bac3-42cd-8a21-c17581035614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:58.495432 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 989/1959] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:58.499887 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111d3a3-d96c-4d40-9427-9f2895b6f31d None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:58.510635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111d3a3-d96c-4d40-9427-9f2895b6f31d None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:58.515495 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb19434-44ba-4138-8af1-974f5391aadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:58.516174 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb19434-44ba-4138-8af1-974f5391aadb admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:58.516461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb19434-44ba-4138-8af1-974f5391aadb admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:58.516741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb19434-44ba-4138-8af1-974f5391aadb admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:58.517493 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb19434-44ba-4138-8af1-974f5391aadb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:48:58.517843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb19434-44ba-4138-8af1-974f5391aadb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:58.519460 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb19434-44ba-4138-8af1-974f5391aadb admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:58.532416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 971/1960] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:48:58.537906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab63064-bac3-42cd-8a21-c17581035614 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:58.547799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab63064-bac3-42cd-8a21-c17581035614 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:58.553500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9cdf2-668c-4fb7-b578-521f1cabff28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:58.554252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9cdf2-668c-4fb7-b578-521f1cabff28 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:58.554539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9cdf2-668c-4fb7-b578-521f1cabff28 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:58.554814 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9cdf2-668c-4fb7-b578-521f1cabff28 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:58.555525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9cdf2-668c-4fb7-b578-521f1cabff28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:58.557541 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9cdf2-668c-4fb7-b578-521f1cabff28 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:58.590576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 990/1961] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:58 2022] POST /identity/v3/projects => generated 391 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:58.594985 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb19434-44ba-4138-8af1-974f5391aadb admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:58.606258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb19434-44ba-4138-8af1-974f5391aadb admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:58.669760 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02535508-81ed-450d-a969-711a45bc5fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:58.670461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02535508-81ed-450d-a969-711a45bc5fb7 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:58.670727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02535508-81ed-450d-a969-711a45bc5fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:58.671001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02535508-81ed-450d-a969-711a45bc5fb7 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:58.672041 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02535508-81ed-450d-a969-711a45bc5fb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:58.674013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02535508-81ed-450d-a969-711a45bc5fb7 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:58.705355 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 972/1962] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:58 2022] POST /identity/v3/users => generated 296 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:58.710350 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9cdf2-668c-4fb7-b578-521f1cabff28 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:58.722807 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9cdf2-668c-4fb7-b578-521f1cabff28 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:58.730390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852995da-e914-47d6-a956-cce34a737524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:58.731439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852995da-e914-47d6-a956-cce34a737524 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:58.732190 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852995da-e914-47d6-a956-cce34a737524 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:58.733000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852995da-e914-47d6-a956-cce34a737524 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:58.734128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852995da-e914-47d6-a956-cce34a737524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:58.736011 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852995da-e914-47d6-a956-cce34a737524 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:58.760992 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 991/1963] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:58.765262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02535508-81ed-450d-a969-711a45bc5fb7 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:58.778169 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02535508-81ed-450d-a969-711a45bc5fb7 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:58.793665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c673ea-7a44-4e0c-859e-a7a127709f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:58.794678 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c673ea-7a44-4e0c-859e-a7a127709f67 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:58.795104 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c673ea-7a44-4e0c-859e-a7a127709f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:58.795390 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c673ea-7a44-4e0c-859e-a7a127709f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:58.796144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c673ea-7a44-4e0c-859e-a7a127709f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:58.806367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c673ea-7a44-4e0c-859e-a7a127709f67 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:58.823442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 973/1964] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:58 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:58.827987 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852995da-e914-47d6-a956-cce34a737524 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:58.842224 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852995da-e914-47d6-a956-cce34a737524 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:58.855041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea254ec-42cf-4bfc-ad21-66da5b944d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:58.856590 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea254ec-42cf-4bfc-ad21-66da5b944d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:58.856956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea254ec-42cf-4bfc-ad21-66da5b944d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:58.857268 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea254ec-42cf-4bfc-ad21-66da5b944d0a admin admin] PATH_INFO: `/v3/projects/68aae743fe194a089ea9f28330065b39/users/45e485189b454e0e917a3ac7cd121350/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:58.882369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea254ec-42cf-4bfc-ad21-66da5b944d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=68aae743fe194a089ea9f28330065b39, user_id=45e485189b454e0e917a3ac7cd121350, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:58.885124 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea254ec-42cf-4bfc-ad21-66da5b944d0a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:58.903444 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 992/1965] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:58 2022] PUT /identity/v3/projects/68aae743fe194a089ea9f28330065b39/users/45e485189b454e0e917a3ac7cd121350/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:58.918741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d334cbf-0bcb-4353-8813-476814853276 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:58.919212 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d334cbf-0bcb-4353-8813-476814853276 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:58.919498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d334cbf-0bcb-4353-8813-476814853276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:58.964514 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-3d334cbf-0bcb-4353-8813-476814853276 None None] MFA Rules not processed for user `45e485189b454e0e917a3ac7cd121350`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:58.965834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3d334cbf-0bcb-4353-8813-476814853276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:59.045174 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 974/1966] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:48:59.050966 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea254ec-42cf-4bfc-ad21-66da5b944d0a admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:59.128250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea254ec-42cf-4bfc-ad21-66da5b944d0a admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:59.134078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb1d8c-d254-4bf3-922a-feda33a363dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:59.134319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb1d8c-d254-4bf3-922a-feda33a363dc admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:59.134578 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb1d8c-d254-4bf3-922a-feda33a363dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:59.134868 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb1d8c-d254-4bf3-922a-feda33a363dc admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:59.135749 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb1d8c-d254-4bf3-922a-feda33a363dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:59.138048 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb1d8c-d254-4bf3-922a-feda33a363dc admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:59.171101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 993/1967] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:48:59 2022] POST /identity/v3/projects => generated 389 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:59.176884 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d334cbf-0bcb-4353-8813-476814853276 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:59.193677 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d334cbf-0bcb-4353-8813-476814853276 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:59.274800 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:59.295552 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c936e2-2f50-4f8c-85dd-44bb8ff7eece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:59.296493 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c936e2-2f50-4f8c-85dd-44bb8ff7eece admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:59.296786 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c936e2-2f50-4f8c-85dd-44bb8ff7eece admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:59.297456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c936e2-2f50-4f8c-85dd-44bb8ff7eece admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:59.297774 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c936e2-2f50-4f8c-85dd-44bb8ff7eece admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:59.299904 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c936e2-2f50-4f8c-85dd-44bb8ff7eece admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:59.332853 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 975/1968] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:48:59 2022] POST /identity/v3/users => generated 301 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:48:59.334836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb1d8c-d254-4bf3-922a-feda33a363dc admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:59.362748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb1d8c-d254-4bf3-922a-feda33a363dc admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:59.371231 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bcc10-63a5-4a06-8368-336c7200f0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:59.372017 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bcc10-63a5-4a06-8368-336c7200f0de admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:59.372500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bcc10-63a5-4a06-8368-336c7200f0de admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:59.372790 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bcc10-63a5-4a06-8368-336c7200f0de admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:59.373586 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4bcc10-63a5-4a06-8368-336c7200f0de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:59.375440 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4bcc10-63a5-4a06-8368-336c7200f0de admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:59.391756 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 994/1969] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:48:59.393972 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c936e2-2f50-4f8c-85dd-44bb8ff7eece admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:59.396005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-37c936e2-2f50-4f8c-85dd-44bb8ff7eece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:59.397401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bcc10-63a5-4a06-8368-336c7200f0de admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:59.408969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bcc10-63a5-4a06-8368-336c7200f0de admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:59.410010 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c936e2-2f50-4f8c-85dd-44bb8ff7eece admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:59.414150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ffcdd-5166-4970-b17a-47a81dfded8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:59.415020 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f8267-b50c-4d22-aca6-f8c4affafa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:59.415143 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ffcdd-5166-4970-b17a-47a81dfded8b admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:59.415338 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ffcdd-5166-4970-b17a-47a81dfded8b admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:59.415778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ffcdd-5166-4970-b17a-47a81dfded8b admin admin] PATH_INFO: `/v3/projects/0da47562c4054af586810c4f47883ccc/users/0071c867a4fe498780efb7bcd774e131/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:59.416070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f8267-b50c-4d22-aca6-f8c4affafa48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:59.416440 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f8267-b50c-4d22-aca6-f8c4affafa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:59.417819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f8267-b50c-4d22-aca6-f8c4affafa48 admin admin] PATH_INFO: `/v3/users/20d89b1e5f194b3b9081b474f3ecb51a` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:59.419596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f8267-b50c-4d22-aca6-f8c4affafa48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20d89b1e5f194b3b9081b474f3ecb51a)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:59.422095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f8267-b50c-4d22-aca6-f8c4affafa48 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:59.433713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ffcdd-5166-4970-b17a-47a81dfded8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0da47562c4054af586810c4f47883ccc, user_id=0071c867a4fe498780efb7bcd774e131, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:59.436638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ffcdd-5166-4970-b17a-47a81dfded8b admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:59.448369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:59.454014 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 995/1970] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:59 2022] PUT /identity/v3/projects/0da47562c4054af586810c4f47883ccc/users/0071c867a4fe498780efb7bcd774e131/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:48:59.459095 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ffcdd-5166-4970-b17a-47a81dfded8b admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:59.484013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ba4f8267-b50c-4d22-aca6-f8c4affafa48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20d89b1e5f194b3b9081b474f3ecb51a'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:59.500342 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ba4f8267-b50c-4d22-aca6-f8c4affafa48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20d89b1e5f194b3b9081b474f3ecb51a'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:59.511781 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ba4f8267-b50c-4d22-aca6-f8c4affafa48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20d89b1e5f194b3b9081b474f3ecb51a'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:59.512624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ba4f8267-b50c-4d22-aca6-f8c4affafa48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20d89b1e5f194b3b9081b474f3ecb51a'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:59.513060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ba4f8267-b50c-4d22-aca6-f8c4affafa48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20d89b1e5f194b3b9081b474f3ecb51a'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:48:59.530768 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 976/1971] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:48:59 2022] DELETE /identity/v3/users/20d89b1e5f194b3b9081b474f3ecb51a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:48:59.536376 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216355da-94c8-4fc7-87b2-7859eb2b506a None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:59.536752 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216355da-94c8-4fc7-87b2-7859eb2b506a None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:59.537156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216355da-94c8-4fc7-87b2-7859eb2b506a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:59.538512 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ffcdd-5166-4970-b17a-47a81dfded8b admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:59.557396 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-216355da-94c8-4fc7-87b2-7859eb2b506a None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:59.558940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-216355da-94c8-4fc7-87b2-7859eb2b506a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:59.607943 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1aa2d-d422-4399-8c40-f86c70b00cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:59.608627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1aa2d-d422-4399-8c40-f86c70b00cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:59.608993 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1aa2d-d422-4399-8c40-f86c70b00cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:59.609338 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1aa2d-d422-4399-8c40-f86c70b00cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:59.610845 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1aa2d-d422-4399-8c40-f86c70b00cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:59.613494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1aa2d-d422-4399-8c40-f86c70b00cc4 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:59.631032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 996/1972] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:48:59 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:48:59.631826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 977/1973] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:48:59.632114 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1aa2d-d422-4399-8c40-f86c70b00cc4 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:59.633863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-65a1aa2d-d422-4399-8c40-f86c70b00cc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:59.643167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216355da-94c8-4fc7-87b2-7859eb2b506a None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:59.645406 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-216355da-94c8-4fc7-87b2-7859eb2b506a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:59.649128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1aa2d-d422-4399-8c40-f86c70b00cc4 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:59.658905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f636c15-d077-46be-99ec-19b1c57de99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:59.659963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f636c15-d077-46be-99ec-19b1c57de99a admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:59.660248 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f636c15-d077-46be-99ec-19b1c57de99a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:59.660527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f636c15-d077-46be-99ec-19b1c57de99a admin admin] PATH_INFO: `/v3/projects/0da47562c4054af586810c4f47883ccc/users/0071c867a4fe498780efb7bcd774e131/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:59.663478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f636c15-d077-46be-99ec-19b1c57de99a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0da47562c4054af586810c4f47883ccc, user_id=0071c867a4fe498780efb7bcd774e131, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:59.666254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f636c15-d077-46be-99ec-19b1c57de99a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:59.678989 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 997/1974] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:48:59 2022] PUT /identity/v3/projects/0da47562c4054af586810c4f47883ccc/users/0071c867a4fe498780efb7bcd774e131/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:48:59.696498 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55448e-5a76-4790-b67e-fbcd0317ece4 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:59.696787 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55448e-5a76-4790-b67e-fbcd0317ece4 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:59.697067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55448e-5a76-4790-b67e-fbcd0317ece4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:59.710607 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:59.715311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216355da-94c8-4fc7-87b2-7859eb2b506a None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:59.737890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-6a55448e-5a76-4790-b67e-fbcd0317ece4 None None] MFA Rules not processed for user `0071c867a4fe498780efb7bcd774e131`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:48:59.739222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-6a55448e-5a76-4790-b67e-fbcd0317ece4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:59.782048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49219e4-db60-43c3-9f97-724c3aaf72a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:59.782723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49219e4-db60-43c3-9f97-724c3aaf72a4 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:59.782993 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49219e4-db60-43c3-9f97-724c3aaf72a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:59.783262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49219e4-db60-43c3-9f97-724c3aaf72a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:59.825494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 998/1975] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:48:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:48:59.829227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55448e-5a76-4790-b67e-fbcd0317ece4 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:48:59.830839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-6a55448e-5a76-4790-b67e-fbcd0317ece4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:48:59.857155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:48:59.877411 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49219e4-db60-43c3-9f97-724c3aaf72a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:59.879133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49219e4-db60-43c3-9f97-724c3aaf72a4 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:59.891073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 978/1976] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:48:59.931217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55448e-5a76-4790-b67e-fbcd0317ece4 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:48:59.937098 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ede514-3171-4369-b939-27be009eea36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:48:59.937857 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ede514-3171-4369-b939-27be009eea36 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:48:59.938318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ede514-3171-4369-b939-27be009eea36 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:48:59.939400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ede514-3171-4369-b939-27be009eea36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:48:59.957468 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ede514-3171-4369-b939-27be009eea36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:48:59.962403 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ede514-3171-4369-b939-27be009eea36 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:48:59.981026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 999/1977] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:48:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:49:00.073356 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49219e4-db60-43c3-9f97-724c3aaf72a4 service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:00.083717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49219e4-db60-43c3-9f97-724c3aaf72a4 service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:00.089450 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55192133-4129-4029-9c47-0d4b08325c64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:00.090452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55192133-4129-4029-9c47-0d4b08325c64 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:00.090926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55192133-4129-4029-9c47-0d4b08325c64 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:00.091428 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55192133-4129-4029-9c47-0d4b08325c64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:00.093739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-55192133-4129-4029-9c47-0d4b08325c64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:00.207099 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55192133-4129-4029-9c47-0d4b08325c64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:00.208758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55192133-4129-4029-9c47-0d4b08325c64 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:00.219775 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 979/1978] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:00.357901 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ede514-3171-4369-b939-27be009eea36 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:00.359871 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-51ede514-3171-4369-b939-27be009eea36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:00.387622 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ede514-3171-4369-b939-27be009eea36 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:00.396999 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905888b4-5cda-4d7d-aecf-c77f0f5bd1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:00.398228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905888b4-5cda-4d7d-aecf-c77f0f5bd1d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:00.399923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905888b4-5cda-4d7d-aecf-c77f0f5bd1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:00.400279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905888b4-5cda-4d7d-aecf-c77f0f5bd1d6 admin admin] PATH_INFO: `/v3/projects/ffbdc6bee1c744918f6d6aa74d45304b` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:00.401975 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905888b4-5cda-4d7d-aecf-c77f0f5bd1d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffbdc6bee1c744918f6d6aa74d45304b)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:00.404328 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905888b4-5cda-4d7d-aecf-c77f0f5bd1d6 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:00.476332 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-905888b4-5cda-4d7d-aecf-c77f0f5bd1d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffbdc6bee1c744918f6d6aa74d45304b', 'request_id': 'req-905888b4-5cda-4d7d-aecf-c77f0f5bd1d6'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:00.484687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-905888b4-5cda-4d7d-aecf-c77f0f5bd1d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffbdc6bee1c744918f6d6aa74d45304b', 'request_id': 'req-905888b4-5cda-4d7d-aecf-c77f0f5bd1d6'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:00.494244 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-905888b4-5cda-4d7d-aecf-c77f0f5bd1d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffbdc6bee1c744918f6d6aa74d45304b', 'request_id': 'req-905888b4-5cda-4d7d-aecf-c77f0f5bd1d6'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:00.517498 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-905888b4-5cda-4d7d-aecf-c77f0f5bd1d6 admin admin] The token cache is being invalidate because project ffbdc6bee1c744918f6d6aa74d45304b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:00.518160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-905888b4-5cda-4d7d-aecf-c77f0f5bd1d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:00.520008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-905888b4-5cda-4d7d-aecf-c77f0f5bd1d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:00.522689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1000/1979] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:00 2022] DELETE /identity/v3/projects/ffbdc6bee1c744918f6d6aa74d45304b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:00.527984 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55192133-4129-4029-9c47-0d4b08325c64 service cinder] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:00.544666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55192133-4129-4029-9c47-0d4b08325c64 service cinder] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:00.615584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:00.626610 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e172d-d959-47ce-a21c-0b83736289ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:00.627661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e172d-d959-47ce-a21c-0b83736289ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:00.628320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e172d-d959-47ce-a21c-0b83736289ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:00.628612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e172d-d959-47ce-a21c-0b83736289ec admin admin] PATH_INFO: `/v3/users/07fbbdbf625b492784c3d57090daa50d` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:00.629896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e172d-d959-47ce-a21c-0b83736289ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=07fbbdbf625b492784c3d57090daa50d)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:00.632075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e172d-d959-47ce-a21c-0b83736289ec admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:00.690213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-025e172d-d959-47ce-a21c-0b83736289ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07fbbdbf625b492784c3d57090daa50d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:00.712259 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-025e172d-d959-47ce-a21c-0b83736289ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07fbbdbf625b492784c3d57090daa50d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:00.724095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-025e172d-d959-47ce-a21c-0b83736289ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07fbbdbf625b492784c3d57090daa50d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:00.724838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-025e172d-d959-47ce-a21c-0b83736289ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07fbbdbf625b492784c3d57090daa50d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:00.725858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-025e172d-d959-47ce-a21c-0b83736289ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07fbbdbf625b492784c3d57090daa50d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:00.764696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 980/1980] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:00 2022] DELETE /identity/v3/users/07fbbdbf625b492784c3d57090daa50d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:01.137377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905888b4-5cda-4d7d-aecf-c77f0f5bd1d6 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:01.139618 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-905888b4-5cda-4d7d-aecf-c77f0f5bd1d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:01.157500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905888b4-5cda-4d7d-aecf-c77f0f5bd1d6 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:01.223444 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e172d-d959-47ce-a21c-0b83736289ec admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:01.225551 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-025e172d-d959-47ce-a21c-0b83736289ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:01.270635 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d300cc4-f64f-4e93-98d8-7cf72948cc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:01.271603 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d300cc4-f64f-4e93-98d8-7cf72948cc12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:01.271892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d300cc4-f64f-4e93-98d8-7cf72948cc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:01.272195 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d300cc4-f64f-4e93-98d8-7cf72948cc12 admin admin] PATH_INFO: `/v3/users/4e86862914304673a384443dc6713650` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:01.273734 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d300cc4-f64f-4e93-98d8-7cf72948cc12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e86862914304673a384443dc6713650)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:01.276250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d300cc4-f64f-4e93-98d8-7cf72948cc12 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:01.319846 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e172d-d959-47ce-a21c-0b83736289ec admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:01.325680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cdbdd-673d-4c29-8b2c-6b791ceaf23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:01.326692 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cdbdd-673d-4c29-8b2c-6b791ceaf23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:01.326983 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cdbdd-673d-4c29-8b2c-6b791ceaf23b admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:01.327556 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cdbdd-673d-4c29-8b2c-6b791ceaf23b admin admin] PATH_INFO: `/v3/projects/a36d24d8aaca4f9a86e7dd3d3a7bef2c` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:01.329334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18cdbdd-673d-4c29-8b2c-6b791ceaf23b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a36d24d8aaca4f9a86e7dd3d3a7bef2c)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:01.333330 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18cdbdd-673d-4c29-8b2c-6b791ceaf23b admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:01.344940 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:01.349215 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4d300cc4-f64f-4e93-98d8-7cf72948cc12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e86862914304673a384443dc6713650'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:01.377036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4d300cc4-f64f-4e93-98d8-7cf72948cc12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e86862914304673a384443dc6713650'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:01.377829 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4d300cc4-f64f-4e93-98d8-7cf72948cc12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e86862914304673a384443dc6713650'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:01.386754 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4d300cc4-f64f-4e93-98d8-7cf72948cc12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e86862914304673a384443dc6713650'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:01.394067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4d300cc4-f64f-4e93-98d8-7cf72948cc12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e86862914304673a384443dc6713650'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:01.400420 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1001/1981] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:01 2022] DELETE /identity/v3/users/4e86862914304673a384443dc6713650 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:01.402770 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e18cdbdd-673d-4c29-8b2c-6b791ceaf23b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a36d24d8aaca4f9a86e7dd3d3a7bef2c', 'request_id': 'req-e18cdbdd-673d-4c29-8b2c-6b791ceaf23b'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:01.405218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6b6c7-04d4-49ac-979f-2eb5ec4e37da None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:01.405511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6b6c7-04d4-49ac-979f-2eb5ec4e37da None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:01.405831 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6b6c7-04d4-49ac-979f-2eb5ec4e37da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:01.409582 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e18cdbdd-673d-4c29-8b2c-6b791ceaf23b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a36d24d8aaca4f9a86e7dd3d3a7bef2c', 'request_id': 'req-e18cdbdd-673d-4c29-8b2c-6b791ceaf23b'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:01.419341 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e18cdbdd-673d-4c29-8b2c-6b791ceaf23b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a36d24d8aaca4f9a86e7dd3d3a7bef2c', 'request_id': 'req-e18cdbdd-673d-4c29-8b2c-6b791ceaf23b'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:01.426264 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-cfe6b6c7-04d4-49ac-979f-2eb5ec4e37da None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:01.428460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-cfe6b6c7-04d4-49ac-979f-2eb5ec4e37da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:01.434255 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e18cdbdd-673d-4c29-8b2c-6b791ceaf23b admin admin] The token cache is being invalidate because project a36d24d8aaca4f9a86e7dd3d3a7bef2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:01.434553 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e18cdbdd-673d-4c29-8b2c-6b791ceaf23b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:01.435200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e18cdbdd-673d-4c29-8b2c-6b791ceaf23b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:01.437016 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 981/1982] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:01 2022] DELETE /identity/v3/projects/a36d24d8aaca4f9a86e7dd3d3a7bef2c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:49:01.442301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2f095-0d17-4d0e-bbaa-fec0dbb024ae None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:01.442641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2f095-0d17-4d0e-bbaa-fec0dbb024ae None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:01.443625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2f095-0d17-4d0e-bbaa-fec0dbb024ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:01.451455 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:01.469875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-eee2f095-0d17-4d0e-bbaa-fec0dbb024ae None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:01.471253 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-eee2f095-0d17-4d0e-bbaa-fec0dbb024ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:01.506683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1002/1983] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:01.517507 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6b6c7-04d4-49ac-979f-2eb5ec4e37da None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:01.519070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-cfe6b6c7-04d4-49ac-979f-2eb5ec4e37da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:01.570231 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 982/1984] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:01.578067 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26378a8-d30b-42a5-89a0-31ceab5220b3 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:01.578357 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26378a8-d30b-42a5-89a0-31ceab5220b3 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:01.578792 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26378a8-d30b-42a5-89a0-31ceab5220b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:01.594535 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6b6c7-04d4-49ac-979f-2eb5ec4e37da None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:01.598482 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-e26378a8-d30b-42a5-89a0-31ceab5220b3 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:01.599863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e26378a8-d30b-42a5-89a0-31ceab5220b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:01.600629 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf22dae-0088-409d-bc6b-9d1370d0a9ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:01.601678 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf22dae-0088-409d-bc6b-9d1370d0a9ba service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:01.602941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf22dae-0088-409d-bc6b-9d1370d0a9ba service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:01.603225 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf22dae-0088-409d-bc6b-9d1370d0a9ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:01.609032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 983/1985] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:01.613843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26378a8-d30b-42a5-89a0-31ceab5220b3 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:01.619377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf22dae-0088-409d-bc6b-9d1370d0a9ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:01.621629 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf22dae-0088-409d-bc6b-9d1370d0a9ba service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:01.627142 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26378a8-d30b-42a5-89a0-31ceab5220b3 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:01.633865 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9643e48-0ec8-4dc5-9247-0cf10f175f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:01.634552 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9643e48-0ec8-4dc5-9247-0cf10f175f10 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:01.634834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9643e48-0ec8-4dc5-9247-0cf10f175f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:01.635113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9643e48-0ec8-4dc5-9247-0cf10f175f10 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:01.635424 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1003/1986] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:49:01.636581 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9643e48-0ec8-4dc5-9247-0cf10f175f10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:01.636983 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9643e48-0ec8-4dc5-9247-0cf10f175f10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:01.638717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9643e48-0ec8-4dc5-9247-0cf10f175f10 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:01.654564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 984/1987] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:49:01.659731 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf22dae-0088-409d-bc6b-9d1370d0a9ba service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:01.681459 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf22dae-0088-409d-bc6b-9d1370d0a9ba service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:01.688318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b36f2c-55fb-4021-a257-c2907deee4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:01.689798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b36f2c-55fb-4021-a257-c2907deee4a5 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:01.690142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b36f2c-55fb-4021-a257-c2907deee4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:01.690425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b36f2c-55fb-4021-a257-c2907deee4a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:01.691124 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b36f2c-55fb-4021-a257-c2907deee4a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:01.694817 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b36f2c-55fb-4021-a257-c2907deee4a5 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:01.734061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1004/1988] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:01 2022] POST /identity/v3/projects => generated 365 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:01.739223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9643e48-0ec8-4dc5-9247-0cf10f175f10 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:01.760322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9643e48-0ec8-4dc5-9247-0cf10f175f10 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:01.907845 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919732b-fb54-432e-a296-6b5aedb8d4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:01.908891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919732b-fb54-432e-a296-6b5aedb8d4a8 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:01.909218 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919732b-fb54-432e-a296-6b5aedb8d4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:01.909502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919732b-fb54-432e-a296-6b5aedb8d4a8 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:01.910312 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919732b-fb54-432e-a296-6b5aedb8d4a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:01.913278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919732b-fb54-432e-a296-6b5aedb8d4a8 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:01.960944 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 985/1989] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:01 2022] POST /identity/v3/users => generated 283 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:49:01.965150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b36f2c-55fb-4021-a257-c2907deee4a5 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:01.987193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b36f2c-55fb-4021-a257-c2907deee4a5 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:02.005330 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147031fb-08c6-44f0-8815-4d8bf8f3e8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:02.007724 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147031fb-08c6-44f0-8815-4d8bf8f3e8ca admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:02.008017 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147031fb-08c6-44f0-8815-4d8bf8f3e8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:02.008343 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147031fb-08c6-44f0-8815-4d8bf8f3e8ca admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:02.009368 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147031fb-08c6-44f0-8815-4d8bf8f3e8ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:02.011220 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147031fb-08c6-44f0-8815-4d8bf8f3e8ca admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:02.034353 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919732b-fb54-432e-a296-6b5aedb8d4a8 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:02.036374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-2919732b-fb54-432e-a296-6b5aedb8d4a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:02.062542 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919732b-fb54-432e-a296-6b5aedb8d4a8 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:02.069567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568810a3-18da-4eba-a9b4-52ee3693f6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:02.071078 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568810a3-18da-4eba-a9b4-52ee3693f6b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:02.071602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568810a3-18da-4eba-a9b4-52ee3693f6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:02.072638 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568810a3-18da-4eba-a9b4-52ee3693f6b0 admin admin] PATH_INFO: `/v3/users/4aa2f60039e84581b7f9715a68e691a1` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:02.074825 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568810a3-18da-4eba-a9b4-52ee3693f6b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aa2f60039e84581b7f9715a68e691a1)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:02.077400 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568810a3-18da-4eba-a9b4-52ee3693f6b0 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:02.109794 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1005/1990] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:02.111065 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147031fb-08c6-44f0-8815-4d8bf8f3e8ca admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:02.129597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147031fb-08c6-44f0-8815-4d8bf8f3e8ca admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:02.136103 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d59484-3e36-4f6b-8faf-e7a88b2f0cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:02.136862 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d59484-3e36-4f6b-8faf-e7a88b2f0cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:02.137271 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d59484-3e36-4f6b-8faf-e7a88b2f0cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:02.137594 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d59484-3e36-4f6b-8faf-e7a88b2f0cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:02.138912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d59484-3e36-4f6b-8faf-e7a88b2f0cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:02.140680 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d59484-3e36-4f6b-8faf-e7a88b2f0cf7 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:02.159757 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1006/1991] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:02.165359 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d59484-3e36-4f6b-8faf-e7a88b2f0cf7 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:02.191276 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d59484-3e36-4f6b-8faf-e7a88b2f0cf7 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:02.195738 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-568810a3-18da-4eba-a9b4-52ee3693f6b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aa2f60039e84581b7f9715a68e691a1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:02.204544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623ae9a-cfe9-4703-8868-4966c060a625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:02.206605 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623ae9a-cfe9-4703-8868-4966c060a625 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:02.207275 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623ae9a-cfe9-4703-8868-4966c060a625 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:02.207621 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623ae9a-cfe9-4703-8868-4966c060a625 admin admin] PATH_INFO: `/v3/projects/b353b54cb4dc4b78bf02e2207f30be3d/users/cfed562a94774067befe9a30b0e1b947/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:02.223026 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-568810a3-18da-4eba-a9b4-52ee3693f6b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aa2f60039e84581b7f9715a68e691a1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:02.230583 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:02.243300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-568810a3-18da-4eba-a9b4-52ee3693f6b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aa2f60039e84581b7f9715a68e691a1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:02.244280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-568810a3-18da-4eba-a9b4-52ee3693f6b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aa2f60039e84581b7f9715a68e691a1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:02.244993 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-568810a3-18da-4eba-a9b4-52ee3693f6b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aa2f60039e84581b7f9715a68e691a1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:02.246833 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623ae9a-cfe9-4703-8868-4966c060a625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b353b54cb4dc4b78bf02e2207f30be3d, user_id=cfed562a94774067befe9a30b0e1b947, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:02.253740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623ae9a-cfe9-4703-8868-4966c060a625 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:02.263152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:02.276787 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 986/1992] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:02 2022] DELETE /identity/v3/users/4aa2f60039e84581b7f9715a68e691a1 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:02.281709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecf5eb-2fc9-4ca8-b8bc-fbf9b12614c8 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:02.282257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecf5eb-2fc9-4ca8-b8bc-fbf9b12614c8 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:02.282608 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecf5eb-2fc9-4ca8-b8bc-fbf9b12614c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:02.287669 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1007/1993] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:02 2022] PUT /identity/v3/projects/b353b54cb4dc4b78bf02e2207f30be3d/users/cfed562a94774067befe9a30b0e1b947/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:02.301065 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f81444-48d5-465f-83af-ed996679cf19 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:02.301381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f81444-48d5-465f-83af-ed996679cf19 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:02.301662 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f81444-48d5-465f-83af-ed996679cf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:02.321531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-32ecf5eb-2fc9-4ca8-b8bc-fbf9b12614c8 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:02.323258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-32ecf5eb-2fc9-4ca8-b8bc-fbf9b12614c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:02.362055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-97f81444-48d5-465f-83af-ed996679cf19 None None] MFA Rules not processed for user `cfed562a94774067befe9a30b0e1b947`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:02.363791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-97f81444-48d5-465f-83af-ed996679cf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:02.502755 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 987/1994] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:02.547047 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecf5eb-2fc9-4ca8-b8bc-fbf9b12614c8 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:02.548522 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-32ecf5eb-2fc9-4ca8-b8bc-fbf9b12614c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:02.593217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1008/1995] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:02.594345 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f81444-48d5-465f-83af-ed996679cf19 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:02.598004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-97f81444-48d5-465f-83af-ed996679cf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:02.633856 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f81444-48d5-465f-83af-ed996679cf19 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:02.647234 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb232e-9548-4447-9a9a-8c824a8708c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:02.648392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb232e-9548-4447-9a9a-8c824a8708c9 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:02.648969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb232e-9548-4447-9a9a-8c824a8708c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:02.649602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb232e-9548-4447-9a9a-8c824a8708c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:02.653104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb232e-9548-4447-9a9a-8c824a8708c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:02.659120 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb232e-9548-4447-9a9a-8c824a8708c9 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:02.670730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecf5eb-2fc9-4ca8-b8bc-fbf9b12614c8 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:02.678002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92060aef-bd1a-406c-8128-c49dd559571c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:02.679645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92060aef-bd1a-406c-8128-c49dd559571c service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:02.680209 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92060aef-bd1a-406c-8128-c49dd559571c service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:02.680634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92060aef-bd1a-406c-8128-c49dd559571c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:02.698948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92060aef-bd1a-406c-8128-c49dd559571c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:02.702946 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92060aef-bd1a-406c-8128-c49dd559571c service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:02.722809 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 988/1996] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:49:02.724629 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92060aef-bd1a-406c-8128-c49dd559571c service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:02.726848 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-92060aef-bd1a-406c-8128-c49dd559571c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:02.746349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92060aef-bd1a-406c-8128-c49dd559571c service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:02.750902 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07facb-1bc9-4a56-9315-ea812af0f760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:02.751678 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07facb-1bc9-4a56-9315-ea812af0f760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:02.752060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07facb-1bc9-4a56-9315-ea812af0f760 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:02.752417 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07facb-1bc9-4a56-9315-ea812af0f760 admin admin] PATH_INFO: `/v3/projects/2dfc884612e447e8abe3c44036cf8831` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:02.753433 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1009/1997] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:02 2022] POST /identity/v3/projects => generated 367 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:49:02.758992 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb232e-9548-4447-9a9a-8c824a8708c9 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:02.759345 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07facb-1bc9-4a56-9315-ea812af0f760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dfc884612e447e8abe3c44036cf8831)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:02.761534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07facb-1bc9-4a56-9315-ea812af0f760 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:02.866707 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2b07facb-1bc9-4a56-9315-ea812af0f760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dfc884612e447e8abe3c44036cf8831', 'request_id': 'req-2b07facb-1bc9-4a56-9315-ea812af0f760'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:02.879623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2b07facb-1bc9-4a56-9315-ea812af0f760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dfc884612e447e8abe3c44036cf8831', 'request_id': 'req-2b07facb-1bc9-4a56-9315-ea812af0f760'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:02.894075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2b07facb-1bc9-4a56-9315-ea812af0f760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dfc884612e447e8abe3c44036cf8831', 'request_id': 'req-2b07facb-1bc9-4a56-9315-ea812af0f760'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:02.921121 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2b07facb-1bc9-4a56-9315-ea812af0f760 admin admin] The token cache is being invalidate because project 2dfc884612e447e8abe3c44036cf8831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:02.921721 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2b07facb-1bc9-4a56-9315-ea812af0f760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:02.925839 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2b07facb-1bc9-4a56-9315-ea812af0f760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:02.932858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 989/1998] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:02 2022] DELETE /identity/v3/projects/2dfc884612e447e8abe3c44036cf8831 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:02.934698 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07facb-1bc9-4a56-9315-ea812af0f760 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:02.937515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-2b07facb-1bc9-4a56-9315-ea812af0f760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:02.972378 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:02.978396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb232e-9548-4447-9a9a-8c824a8708c9 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:03.076123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e797044-16d3-4dd2-b2d9-fa26c9a6e5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:03.077312 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e797044-16d3-4dd2-b2d9-fa26c9a6e5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:03.077853 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e797044-16d3-4dd2-b2d9-fa26c9a6e5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:03.078341 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e797044-16d3-4dd2-b2d9-fa26c9a6e5e0 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:03.079963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e797044-16d3-4dd2-b2d9-fa26c9a6e5e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:03.083309 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e797044-16d3-4dd2-b2d9-fa26c9a6e5e0 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:03.088083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07facb-1bc9-4a56-9315-ea812af0f760 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:03.095354 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91eebe-ae26-40e8-86f1-55d392168655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:03.096402 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91eebe-ae26-40e8-86f1-55d392168655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:03.096858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91eebe-ae26-40e8-86f1-55d392168655 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:03.097311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91eebe-ae26-40e8-86f1-55d392168655 admin admin] PATH_INFO: `/v3/users/a760404a55804f659c2a3c26e7266bde` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:03.100237 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91eebe-ae26-40e8-86f1-55d392168655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a760404a55804f659c2a3c26e7266bde)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:03.104093 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91eebe-ae26-40e8-86f1-55d392168655 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:03.111164 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:03.125941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1010/1999] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:02 2022] POST /identity/v3/users => generated 290 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:03.134689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e797044-16d3-4dd2-b2d9-fa26c9a6e5e0 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:03.142494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1e797044-16d3-4dd2-b2d9-fa26c9a6e5e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:03.158771 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e797044-16d3-4dd2-b2d9-fa26c9a6e5e0 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:03.165696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c847ef3-879b-484a-8629-343e5ae0cf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:03.166625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c847ef3-879b-484a-8629-343e5ae0cf2a admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:03.167128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c847ef3-879b-484a-8629-343e5ae0cf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:03.167319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c847ef3-879b-484a-8629-343e5ae0cf2a admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:03.168436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c847ef3-879b-484a-8629-343e5ae0cf2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:03.170401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c847ef3-879b-484a-8629-343e5ae0cf2a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:03.201360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1011/2000] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:03.202380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-0f91eebe-ae26-40e8-86f1-55d392168655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a760404a55804f659c2a3c26e7266bde'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:03.207493 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c847ef3-879b-484a-8629-343e5ae0cf2a admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:03.221690 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-0f91eebe-ae26-40e8-86f1-55d392168655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a760404a55804f659c2a3c26e7266bde'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:03.232434 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c847ef3-879b-484a-8629-343e5ae0cf2a admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:03.236291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-0f91eebe-ae26-40e8-86f1-55d392168655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a760404a55804f659c2a3c26e7266bde'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:03.238725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-0f91eebe-ae26-40e8-86f1-55d392168655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a760404a55804f659c2a3c26e7266bde'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:03.239954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-0f91eebe-ae26-40e8-86f1-55d392168655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a760404a55804f659c2a3c26e7266bde'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:03.240977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379c5dc-767b-4f6f-ac9e-b64f072502da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:03.242392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379c5dc-767b-4f6f-ac9e-b64f072502da admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:03.242927 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379c5dc-767b-4f6f-ac9e-b64f072502da admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:03.243807 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379c5dc-767b-4f6f-ac9e-b64f072502da admin admin] PATH_INFO: `/v3/projects/b3c579ca29a1409ab0b60a586694729e/users/9469b989d0b84d5897106762a5e77c7d/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:03.274289 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 990/2001] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:02 2022] DELETE /identity/v3/users/a760404a55804f659c2a3c26e7266bde => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:03.278354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c379c5dc-767b-4f6f-ac9e-b64f072502da admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c579ca29a1409ab0b60a586694729e, user_id=9469b989d0b84d5897106762a5e77c7d, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:03.278354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c379c5dc-767b-4f6f-ac9e-b64f072502da admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:03.305361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1012/2002] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:03 2022] PUT /identity/v3/projects/b3c579ca29a1409ab0b60a586694729e/users/9469b989d0b84d5897106762a5e77c7d/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:03.306391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91eebe-ae26-40e8-86f1-55d392168655 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:03.308072 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-0f91eebe-ae26-40e8-86f1-55d392168655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:03.423583 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91eebe-ae26-40e8-86f1-55d392168655 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:03.478746 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:03.523100 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379c5dc-767b-4f6f-ac9e-b64f072502da admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:03.525964 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-c379c5dc-767b-4f6f-ac9e-b64f072502da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:03.532484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500cddf-fe8b-4b65-b73f-43ddd98def13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:03.533391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500cddf-fe8b-4b65-b73f-43ddd98def13 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:03.533811 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500cddf-fe8b-4b65-b73f-43ddd98def13 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:03.534302 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500cddf-fe8b-4b65-b73f-43ddd98def13 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:03.551266 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379c5dc-767b-4f6f-ac9e-b64f072502da admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:03.560926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500cddf-fe8b-4b65-b73f-43ddd98def13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:03.567809 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500cddf-fe8b-4b65-b73f-43ddd98def13 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:03.580670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed75f2-d2a7-4341-9efe-05e757d5f799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:03.582614 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed75f2-d2a7-4341-9efe-05e757d5f799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:03.583063 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed75f2-d2a7-4341-9efe-05e757d5f799 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:03.585350 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed75f2-d2a7-4341-9efe-05e757d5f799 admin admin] PATH_INFO: `/v3/projects/77a747d7554f4c07b9a1bc41b45af171` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:03.588375 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed75f2-d2a7-4341-9efe-05e757d5f799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77a747d7554f4c07b9a1bc41b45af171)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:03.590858 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed75f2-d2a7-4341-9efe-05e757d5f799 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:03.597504 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 991/2003] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:03.601817 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500cddf-fe8b-4b65-b73f-43ddd98def13 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:03.635820 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500cddf-fe8b-4b65-b73f-43ddd98def13 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:03.652418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc96181-f78e-43fd-b26f-22ae5315210b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:03.654380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc96181-f78e-43fd-b26f-22ae5315210b admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:03.656145 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc96181-f78e-43fd-b26f-22ae5315210b admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:03.659433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc96181-f78e-43fd-b26f-22ae5315210b admin admin] PATH_INFO: `/v3/projects/b3c579ca29a1409ab0b60a586694729e/users/9469b989d0b84d5897106762a5e77c7d/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:03.666571 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc96181-f78e-43fd-b26f-22ae5315210b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c579ca29a1409ab0b60a586694729e, user_id=9469b989d0b84d5897106762a5e77c7d, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:03.678685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-35ed75f2-d2a7-4341-9efe-05e757d5f799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77a747d7554f4c07b9a1bc41b45af171', 'request_id': 'req-35ed75f2-d2a7-4341-9efe-05e757d5f799'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:03.679992 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc96181-f78e-43fd-b26f-22ae5315210b admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:03.694637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-35ed75f2-d2a7-4341-9efe-05e757d5f799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77a747d7554f4c07b9a1bc41b45af171', 'request_id': 'req-35ed75f2-d2a7-4341-9efe-05e757d5f799'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:03.722400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-35ed75f2-d2a7-4341-9efe-05e757d5f799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77a747d7554f4c07b9a1bc41b45af171', 'request_id': 'req-35ed75f2-d2a7-4341-9efe-05e757d5f799'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:03.733331 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 992/2004] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:03 2022] PUT /identity/v3/projects/b3c579ca29a1409ab0b60a586694729e/users/9469b989d0b84d5897106762a5e77c7d/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:03.743873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-35ed75f2-d2a7-4341-9efe-05e757d5f799 admin admin] The token cache is being invalidate because project 77a747d7554f4c07b9a1bc41b45af171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:03.744214 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-35ed75f2-d2a7-4341-9efe-05e757d5f799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:03.745173 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-35ed75f2-d2a7-4341-9efe-05e757d5f799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:03.747146 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ea85b-8f06-4e09-9a75-652024067f73 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:03.747841 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ea85b-8f06-4e09-9a75-652024067f73 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:03.747841 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ea85b-8f06-4e09-9a75-652024067f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:03.749233 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1013/2005] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:03 2022] DELETE /identity/v3/projects/77a747d7554f4c07b9a1bc41b45af171 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:03.755830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29227dd6-85a3-4e4f-8f99-18ca462ca6ef None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:03.755982 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29227dd6-85a3-4e4f-8f99-18ca462ca6ef None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:03.756288 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29227dd6-85a3-4e4f-8f99-18ca462ca6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:03.778379 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-29227dd6-85a3-4e4f-8f99-18ca462ca6ef None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:03.779654 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-29227dd6-85a3-4e4f-8f99-18ca462ca6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:03.810680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-c94ea85b-8f06-4e09-9a75-652024067f73 None None] MFA Rules not processed for user `9469b989d0b84d5897106762a5e77c7d`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:03.813252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-c94ea85b-8f06-4e09-9a75-652024067f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:03.869783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1014/2006] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:03.875899 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fc365-5fe9-4acf-b84f-8961ea773328 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:03.876284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fc365-5fe9-4acf-b84f-8961ea773328 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:03.876571 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fc365-5fe9-4acf-b84f-8961ea773328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:03.918011 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-656fc365-5fe9-4acf-b84f-8961ea773328 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:03.920310 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-656fc365-5fe9-4acf-b84f-8961ea773328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:03.953820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1015/2007] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:03.953820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fc365-5fe9-4acf-b84f-8961ea773328 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:03.954637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-656fc365-5fe9-4acf-b84f-8961ea773328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:03.957697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 993/2008] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:03 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:03.959761 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ea85b-8f06-4e09-9a75-652024067f73 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:03.978162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fc365-5fe9-4acf-b84f-8961ea773328 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:03.980693 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ea85b-8f06-4e09-9a75-652024067f73 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:03.982520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809886c-6909-44a4-b782-358d9d3c07d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:03.983927 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809886c-6909-44a4-b782-358d9d3c07d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:03.984290 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809886c-6909-44a4-b782-358d9d3c07d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:03.984638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809886c-6909-44a4-b782-358d9d3c07d6 admin admin] PATH_INFO: `/v3/projects/c3f28feb32b04215b6dc42ed33afa582` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:03.986640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809886c-6909-44a4-b782-358d9d3c07d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3f28feb32b04215b6dc42ed33afa582)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:03.987486 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c5522-3b64-421d-8999-0c590451d02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:03.988267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c5522-3b64-421d-8999-0c590451d02f admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:03.988378 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4809886c-6909-44a4-b782-358d9d3c07d6 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:03.988690 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c5522-3b64-421d-8999-0c590451d02f admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:03.989130 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c5522-3b64-421d-8999-0c590451d02f admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:03.990365 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c5522-3b64-421d-8999-0c590451d02f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:03.990806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c5522-3b64-421d-8999-0c590451d02f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:03.992505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c5522-3b64-421d-8999-0c590451d02f admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:04.017451 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 994/2009] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:49:04.019326 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c5522-3b64-421d-8999-0c590451d02f admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:04.023385 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-6b4c5522-3b64-421d-8999-0c590451d02f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:04.046520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:04.067881 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4809886c-6909-44a4-b782-358d9d3c07d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3f28feb32b04215b6dc42ed33afa582', 'request_id': 'req-4809886c-6909-44a4-b782-358d9d3c07d6'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:04.078892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4809886c-6909-44a4-b782-358d9d3c07d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3f28feb32b04215b6dc42ed33afa582', 'request_id': 'req-4809886c-6909-44a4-b782-358d9d3c07d6'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:04.088525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4809886c-6909-44a4-b782-358d9d3c07d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3f28feb32b04215b6dc42ed33afa582', 'request_id': 'req-4809886c-6909-44a4-b782-358d9d3c07d6'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:04.107573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4809886c-6909-44a4-b782-358d9d3c07d6 admin admin] The token cache is being invalidate because project c3f28feb32b04215b6dc42ed33afa582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:04.107935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4809886c-6909-44a4-b782-358d9d3c07d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:04.108673 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4809886c-6909-44a4-b782-358d9d3c07d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:04.111045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1016/2010] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:03 2022] DELETE /identity/v3/projects/c3f28feb32b04215b6dc42ed33afa582 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:04.112116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809886c-6909-44a4-b782-358d9d3c07d6 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:04.114003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-4809886c-6909-44a4-b782-358d9d3c07d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:04.117459 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c5522-3b64-421d-8999-0c590451d02f admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:04.186818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:04.193803 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27220f7-6dac-496d-98fc-bdc24c78ff51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:04.196367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27220f7-6dac-496d-98fc-bdc24c78ff51 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:04.197249 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27220f7-6dac-496d-98fc-bdc24c78ff51 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:04.197388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27220f7-6dac-496d-98fc-bdc24c78ff51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:04.199594 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-c27220f7-6dac-496d-98fc-bdc24c78ff51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:04.212965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4809886c-6909-44a4-b782-358d9d3c07d6 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:04.218215 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083e244-45f2-4b21-8023-0f37dc17254b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:04.218890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083e244-45f2-4b21-8023-0f37dc17254b admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:04.219188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083e244-45f2-4b21-8023-0f37dc17254b admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:04.219439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083e244-45f2-4b21-8023-0f37dc17254b admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:04.220132 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083e244-45f2-4b21-8023-0f37dc17254b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:04.222094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083e244-45f2-4b21-8023-0f37dc17254b admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:04.261708 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1017/2011] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:04 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:04.263913 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf24874-ae2a-4cee-a830-5fa84d106c8e None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:04.264559 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf24874-ae2a-4cee-a830-5fa84d106c8e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:04.265032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf24874-ae2a-4cee-a830-5fa84d106c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:04.289455 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27220f7-6dac-496d-98fc-bdc24c78ff51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:04.291062 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27220f7-6dac-496d-98fc-bdc24c78ff51 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:04.296017 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-aaf24874-ae2a-4cee-a830-5fa84d106c8e None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:04.298102 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-aaf24874-ae2a-4cee-a830-5fa84d106c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:04.304439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 995/2012] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:04 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:04.306835 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27220f7-6dac-496d-98fc-bdc24c78ff51 service cinder] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:04.322978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27220f7-6dac-496d-98fc-bdc24c78ff51 service cinder] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:04.401221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1018/2013] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:04.412426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171e467-0c58-4d30-a886-01e85e108758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:04.414498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171e467-0c58-4d30-a886-01e85e108758 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:04.415357 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171e467-0c58-4d30-a886-01e85e108758 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:04.415832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171e467-0c58-4d30-a886-01e85e108758 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:04.418648 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171e467-0c58-4d30-a886-01e85e108758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:04.419576 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171e467-0c58-4d30-a886-01e85e108758 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:04.420033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d910abd-2179-425b-a026-0a0cfaa3a21f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:04.420360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d910abd-2179-425b-a026-0a0cfaa3a21f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:04.420663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d910abd-2179-425b-a026-0a0cfaa3a21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:04.447504 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-1d910abd-2179-425b-a026-0a0cfaa3a21f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:04.447806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 996/2014] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:04 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:04.452537 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171e467-0c58-4d30-a886-01e85e108758 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:04.453822 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1d910abd-2179-425b-a026-0a0cfaa3a21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:04.468915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1019/2015] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:04.470813 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d910abd-2179-425b-a026-0a0cfaa3a21f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:04.474743 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171e467-0c58-4d30-a886-01e85e108758 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:04.481838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd0edf-a918-4243-ade8-015297c0b378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:04.482456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd0edf-a918-4243-ade8-015297c0b378 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:04.482744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd0edf-a918-4243-ade8-015297c0b378 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:04.483062 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd0edf-a918-4243-ade8-015297c0b378 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:04.483801 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd0edf-a918-4243-ade8-015297c0b378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:04.485526 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd0edf-a918-4243-ade8-015297c0b378 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:04.486023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d910abd-2179-425b-a026-0a0cfaa3a21f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:04.494154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f735f38-a576-404a-87cf-584eba00ab87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:04.494844 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f735f38-a576-404a-87cf-584eba00ab87 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:04.495123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f735f38-a576-404a-87cf-584eba00ab87 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:04.495402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f735f38-a576-404a-87cf-584eba00ab87 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:04.496143 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f735f38-a576-404a-87cf-584eba00ab87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:04.496454 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f735f38-a576-404a-87cf-584eba00ab87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:04.498101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f735f38-a576-404a-87cf-584eba00ab87 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:04.500222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 997/2016] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:04.504221 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd0edf-a918-4243-ade8-015297c0b378 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:04.516229 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd0edf-a918-4243-ade8-015297c0b378 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:04.524035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada462a4-7f47-484f-89d4-7d1743b2b4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:04.524887 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada462a4-7f47-484f-89d4-7d1743b2b4d8 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:04.525318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada462a4-7f47-484f-89d4-7d1743b2b4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:04.526279 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada462a4-7f47-484f-89d4-7d1743b2b4d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:04.527183 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada462a4-7f47-484f-89d4-7d1743b2b4d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:04.529149 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada462a4-7f47-484f-89d4-7d1743b2b4d8 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:04.531432 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1020/2017] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:49:04.537116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f735f38-a576-404a-87cf-584eba00ab87 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:04.555956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 998/2018] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:49:04.556133 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f735f38-a576-404a-87cf-584eba00ab87 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:04.557763 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada462a4-7f47-484f-89d4-7d1743b2b4d8 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:04.560902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc2814-1e6f-4141-a374-560e4598f31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:04.561671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc2814-1e6f-4141-a374-560e4598f31c admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:04.562040 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc2814-1e6f-4141-a374-560e4598f31c admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:04.562389 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc2814-1e6f-4141-a374-560e4598f31c admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:04.563142 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc2814-1e6f-4141-a374-560e4598f31c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:04.565935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc2814-1e6f-4141-a374-560e4598f31c admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:04.575522 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada462a4-7f47-484f-89d4-7d1743b2b4d8 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:04.581546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57371feb-8d22-4b41-83a8-8b970075cc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:04.582593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57371feb-8d22-4b41-83a8-8b970075cc7e admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:04.582976 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57371feb-8d22-4b41-83a8-8b970075cc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:04.583204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57371feb-8d22-4b41-83a8-8b970075cc7e admin admin] PATH_INFO: `/v3/projects/009dfcdbdd3840189d5222ff1606f4fa/users/364c6c6f607045d988dd55d8f467b027/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:04.603850 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1021/2019] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:04 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:04.608628 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc2814-1e6f-4141-a374-560e4598f31c admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:04.608823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57371feb-8d22-4b41-83a8-8b970075cc7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=009dfcdbdd3840189d5222ff1606f4fa, user_id=364c6c6f607045d988dd55d8f467b027, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:04.611972 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57371feb-8d22-4b41-83a8-8b970075cc7e admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:04.623876 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:04.636708 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 999/2020] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:04 2022] PUT /identity/v3/projects/009dfcdbdd3840189d5222ff1606f4fa/users/364c6c6f607045d988dd55d8f467b027/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:04.648821 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f4b44-a256-4d83-9716-d63f99149b1a None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:04.649404 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f4b44-a256-4d83-9716-d63f99149b1a None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:04.649506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f4b44-a256-4d83-9716-d63f99149b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:04.684191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-3c2f4b44-a256-4d83-9716-d63f99149b1a None None] MFA Rules not processed for user `364c6c6f607045d988dd55d8f467b027`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:04.686302 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3c2f4b44-a256-4d83-9716-d63f99149b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:04.716305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc2814-1e6f-4141-a374-560e4598f31c admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:04.727412 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:04.779653 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1000/2021] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:04.832888 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e981a8-48f7-4f5f-9a43-827ee8445299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:04.835141 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e981a8-48f7-4f5f-9a43-827ee8445299 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:04.835556 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e981a8-48f7-4f5f-9a43-827ee8445299 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:04.835814 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e981a8-48f7-4f5f-9a43-827ee8445299 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:04.836859 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e981a8-48f7-4f5f-9a43-827ee8445299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:04.838966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f4b44-a256-4d83-9716-d63f99149b1a None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:04.839816 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e981a8-48f7-4f5f-9a43-827ee8445299 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:04.840514 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3c2f4b44-a256-4d83-9716-d63f99149b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:04.869819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1022/2022] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:04 2022] POST /identity/v3/users => generated 291 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:49:04.872765 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e981a8-48f7-4f5f-9a43-827ee8445299 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:04.888715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e981a8-48f7-4f5f-9a43-827ee8445299 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:04.897404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10a867-ccfb-45ba-ac66-19bdb88f9197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:04.898386 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10a867-ccfb-45ba-ac66-19bdb88f9197 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:04.898848 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10a867-ccfb-45ba-ac66-19bdb88f9197 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:04.899301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10a867-ccfb-45ba-ac66-19bdb88f9197 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:04.900187 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10a867-ccfb-45ba-ac66-19bdb88f9197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:04.902264 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10a867-ccfb-45ba-ac66-19bdb88f9197 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:04.916591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1023/2023] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:49:04.922384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10a867-ccfb-45ba-ac66-19bdb88f9197 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:04.937427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10a867-ccfb-45ba-ac66-19bdb88f9197 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:04.946358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de25398-c640-41b6-9209-9a609da8a44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:04.947226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de25398-c640-41b6-9209-9a609da8a44e admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:04.947687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de25398-c640-41b6-9209-9a609da8a44e admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:04.948499 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de25398-c640-41b6-9209-9a609da8a44e admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:04.949664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de25398-c640-41b6-9209-9a609da8a44e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:04.951921 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de25398-c640-41b6-9209-9a609da8a44e admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:04.954727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f4b44-a256-4d83-9716-d63f99149b1a None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:04.963657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e6101-a454-464e-bf47-2351b8cb76cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:04.964882 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e6101-a454-464e-bf47-2351b8cb76cc service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:04.965417 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e6101-a454-464e-bf47-2351b8cb76cc service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:04.965920 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e6101-a454-464e-bf47-2351b8cb76cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:04.968502 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1024/2024] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:04.977295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de25398-c640-41b6-9209-9a609da8a44e admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:04.987877 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e6101-a454-464e-bf47-2351b8cb76cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:04.993084 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e6101-a454-464e-bf47-2351b8cb76cc service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:05.001380 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de25398-c640-41b6-9209-9a609da8a44e admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:05.014358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f6133-d8fa-495b-9459-5b9c687b88c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:05.015691 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f6133-d8fa-495b-9459-5b9c687b88c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:05.016771 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1001/2025] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:49:05.017317 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f6133-d8fa-495b-9459-5b9c687b88c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:05.017835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f6133-d8fa-495b-9459-5b9c687b88c6 admin admin] PATH_INFO: `/v3/projects/7c73b6b08d8d460a8f67e24f6d0e8a2d/users/0219e51f2f404f5e86a1e95400309117/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:05.040233 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f6133-d8fa-495b-9459-5b9c687b88c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c73b6b08d8d460a8f67e24f6d0e8a2d, user_id=0219e51f2f404f5e86a1e95400309117, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:05.042891 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f6133-d8fa-495b-9459-5b9c687b88c6 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:05.073007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1025/2026] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:04 2022] PUT /identity/v3/projects/7c73b6b08d8d460a8f67e24f6d0e8a2d/users/0219e51f2f404f5e86a1e95400309117/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:05.083964 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39284db9-4d37-49ee-bed4-bf0386beabcb None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:05.084508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39284db9-4d37-49ee-bed4-bf0386beabcb None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:05.085853 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39284db9-4d37-49ee-bed4-bf0386beabcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:05.129765 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-39284db9-4d37-49ee-bed4-bf0386beabcb None None] MFA Rules not processed for user `0219e51f2f404f5e86a1e95400309117`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:05.133758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-39284db9-4d37-49ee-bed4-bf0386beabcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:05.271535 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1002/2027] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:05.276437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f6133-d8fa-495b-9459-5b9c687b88c6 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:05.313377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39284db9-4d37-49ee-bed4-bf0386beabcb None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:05.315094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-39284db9-4d37-49ee-bed4-bf0386beabcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:05.463936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f6133-d8fa-495b-9459-5b9c687b88c6 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:05.470900 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4de33a-fa8c-4aa7-a06d-c34cd3c67e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:05.471589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4de33a-fa8c-4aa7-a06d-c34cd3c67e5c admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:05.471935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4de33a-fa8c-4aa7-a06d-c34cd3c67e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:05.472205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4de33a-fa8c-4aa7-a06d-c34cd3c67e5c admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:05.472952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4de33a-fa8c-4aa7-a06d-c34cd3c67e5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:05.474425 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39284db9-4d37-49ee-bed4-bf0386beabcb None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:05.475944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4de33a-fa8c-4aa7-a06d-c34cd3c67e5c admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:05.483157 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:05.496346 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5ff58-2c32-41d3-982c-f3fa6065d692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:05.497574 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5ff58-2c32-41d3-982c-f3fa6065d692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:05.497975 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5ff58-2c32-41d3-982c-f3fa6065d692 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:05.499028 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5ff58-2c32-41d3-982c-f3fa6065d692 admin admin] PATH_INFO: `/v3/users/3e90e6a1993e408cb05d7072c5555d73` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:05.501554 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5ff58-2c32-41d3-982c-f3fa6065d692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e90e6a1993e408cb05d7072c5555d73)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:05.504546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5ff58-2c32-41d3-982c-f3fa6065d692 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:05.512952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:05.539429 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1026/2028] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:05 2022] POST /identity/v3/projects => generated 379 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:05.540342 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4de33a-fa8c-4aa7-a06d-c34cd3c67e5c admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:05.615112 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ebf5ff58-2c32-41d3-982c-f3fa6065d692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e90e6a1993e408cb05d7072c5555d73'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:05.660333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ebf5ff58-2c32-41d3-982c-f3fa6065d692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e90e6a1993e408cb05d7072c5555d73'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:05.676637 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ebf5ff58-2c32-41d3-982c-f3fa6065d692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e90e6a1993e408cb05d7072c5555d73'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:05.677726 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4de33a-fa8c-4aa7-a06d-c34cd3c67e5c admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:05.678452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ebf5ff58-2c32-41d3-982c-f3fa6065d692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e90e6a1993e408cb05d7072c5555d73'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:05.679166 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ebf5ff58-2c32-41d3-982c-f3fa6065d692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e90e6a1993e408cb05d7072c5555d73'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:05.687487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c82237-8fc0-42f9-9026-d012371bb4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:05.688220 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c82237-8fc0-42f9-9026-d012371bb4b0 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:05.688549 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c82237-8fc0-42f9-9026-d012371bb4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:05.691250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c82237-8fc0-42f9-9026-d012371bb4b0 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:05.691999 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c82237-8fc0-42f9-9026-d012371bb4b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:05.694042 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c82237-8fc0-42f9-9026-d012371bb4b0 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:05.710477 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1003/2029] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:05 2022] DELETE /identity/v3/users/3e90e6a1993e408cb05d7072c5555d73 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:05.720280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a12fe8-780d-4b87-8930-9b15d9d4ddd6 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:05.720698 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a12fe8-780d-4b87-8930-9b15d9d4ddd6 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:05.728325 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a12fe8-780d-4b87-8930-9b15d9d4ddd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:05.733764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1027/2030] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:05 2022] POST /identity/v3/users => generated 296 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:05.733764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c82237-8fc0-42f9-9026-d012371bb4b0 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:05.736180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-40c82237-8fc0-42f9-9026-d012371bb4b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:05.752348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-b9a12fe8-780d-4b87-8930-9b15d9d4ddd6 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:05.753764 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-b9a12fe8-780d-4b87-8930-9b15d9d4ddd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:05.754193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c82237-8fc0-42f9-9026-d012371bb4b0 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:05.867171 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbbc18-7a53-4657-8fce-7bcd0cd95bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:05.867611 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1004/2031] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:05.868331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbbc18-7a53-4657-8fce-7bcd0cd95bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:05.868675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbbc18-7a53-4657-8fce-7bcd0cd95bca admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:05.869015 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbbc18-7a53-4657-8fce-7bcd0cd95bca admin admin] PATH_INFO: `/v3/users/45e485189b454e0e917a3ac7cd121350` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:05.869422 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a12fe8-780d-4b87-8930-9b15d9d4ddd6 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:05.871395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacbbc18-7a53-4657-8fce-7bcd0cd95bca admin admin] RBAC: Authorizing `identity:delete_user(user_id=45e485189b454e0e917a3ac7cd121350)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:05.873459 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacbbc18-7a53-4657-8fce-7bcd0cd95bca admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:05.875303 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-b9a12fe8-780d-4b87-8930-9b15d9d4ddd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:05.889093 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a12fe8-780d-4b87-8930-9b15d9d4ddd6 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:05.894381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c116ec5-584f-4115-9a59-6d1248ec6791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:05.895072 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c116ec5-584f-4115-9a59-6d1248ec6791 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:05.895357 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c116ec5-584f-4115-9a59-6d1248ec6791 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:05.896415 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c116ec5-584f-4115-9a59-6d1248ec6791 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:05.897695 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c116ec5-584f-4115-9a59-6d1248ec6791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:05.899542 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c116ec5-584f-4115-9a59-6d1248ec6791 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:05.911696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1005/2032] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:05.916277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c116ec5-584f-4115-9a59-6d1248ec6791 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:05.918862 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-7c116ec5-584f-4115-9a59-6d1248ec6791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:05.961494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-bacbbc18-7a53-4657-8fce-7bcd0cd95bca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45e485189b454e0e917a3ac7cd121350'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:05.976435 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-bacbbc18-7a53-4657-8fce-7bcd0cd95bca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45e485189b454e0e917a3ac7cd121350'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:05.977439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-bacbbc18-7a53-4657-8fce-7bcd0cd95bca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45e485189b454e0e917a3ac7cd121350'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:05.986736 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-bacbbc18-7a53-4657-8fce-7bcd0cd95bca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45e485189b454e0e917a3ac7cd121350'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:05.996854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:06.000845 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-bacbbc18-7a53-4657-8fce-7bcd0cd95bca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45e485189b454e0e917a3ac7cd121350'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:06.002481 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c116ec5-584f-4115-9a59-6d1248ec6791 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:06.009377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95520e8-8715-434e-86a8-d50660c31ea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:06.010060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1028/2033] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:05 2022] DELETE /identity/v3/users/45e485189b454e0e917a3ac7cd121350 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:06.010195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95520e8-8715-434e-86a8-d50660c31ea6 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:06.012471 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbbc18-7a53-4657-8fce-7bcd0cd95bca admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:06.013960 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95520e8-8715-434e-86a8-d50660c31ea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:06.014291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95520e8-8715-434e-86a8-d50660c31ea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:06.014787 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-bacbbc18-7a53-4657-8fce-7bcd0cd95bca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:06.017799 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f95520e8-8715-434e-86a8-d50660c31ea6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:06.038995 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95520e8-8715-434e-86a8-d50660c31ea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:06.040603 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95520e8-8715-434e-86a8-d50660c31ea6 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:06.055860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1006/2034] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:06.057420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fa06e-c707-41f4-bccf-df31246fe78b None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:06.059878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fa06e-c707-41f4-bccf-df31246fe78b None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:06.059878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fa06e-c707-41f4-bccf-df31246fe78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:06.084422 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-763fa06e-c707-41f4-bccf-df31246fe78b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:06.086604 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-763fa06e-c707-41f4-bccf-df31246fe78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:06.106974 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:06.144635 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbbc18-7a53-4657-8fce-7bcd0cd95bca admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:06.152103 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666bad4-f477-480c-ad9a-7d5229f80e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:06.153351 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666bad4-f477-480c-ad9a-7d5229f80e03 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:06.153741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666bad4-f477-480c-ad9a-7d5229f80e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:06.154652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666bad4-f477-480c-ad9a-7d5229f80e03 admin admin] PATH_INFO: `/v3/projects/f77dacc5ddd542128bc051e043b98632/users/36c0f6cd473448058186f5a7e2d6ee17/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:06.207396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666bad4-f477-480c-ad9a-7d5229f80e03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f77dacc5ddd542128bc051e043b98632, user_id=36c0f6cd473448058186f5a7e2d6ee17, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:06.213081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666bad4-f477-480c-ad9a-7d5229f80e03 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:06.215291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1007/2035] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:06.249114 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1029/2036] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:06 2022] PUT /identity/v3/projects/f77dacc5ddd542128bc051e043b98632/users/36c0f6cd473448058186f5a7e2d6ee17/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:06.249327 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fa06e-c707-41f4-bccf-df31246fe78b None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:06.250399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666bad4-f477-480c-ad9a-7d5229f80e03 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:06.250849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-763fa06e-c707-41f4-bccf-df31246fe78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:06.389605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fa06e-c707-41f4-bccf-df31246fe78b None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:06.424936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666bad4-f477-480c-ad9a-7d5229f80e03 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:06.432733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f6a5e-fa0b-4aeb-bc4f-648ba30a8ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:06.433743 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f6a5e-fa0b-4aeb-bc4f-648ba30a8ced admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:06.434267 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f6a5e-fa0b-4aeb-bc4f-648ba30a8ced admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:06.437853 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f6a5e-fa0b-4aeb-bc4f-648ba30a8ced admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:06.437853 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f6a5e-fa0b-4aeb-bc4f-648ba30a8ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:06.450491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f6a5e-fa0b-4aeb-bc4f-648ba30a8ced admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:06.484898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1030/2037] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:49:06.485903 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f6a5e-fa0b-4aeb-bc4f-648ba30a8ced admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:06.519285 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f6a5e-fa0b-4aeb-bc4f-648ba30a8ced admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:06.531622 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a19585-abf7-4744-8833-f69f9292cfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:06.533467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a19585-abf7-4744-8833-f69f9292cfa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:06.533862 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a19585-abf7-4744-8833-f69f9292cfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:06.534334 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a19585-abf7-4744-8833-f69f9292cfa1 admin admin] PATH_INFO: `/v3/projects/f77dacc5ddd542128bc051e043b98632/users/36c0f6cd473448058186f5a7e2d6ee17/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:06.536272 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b15ed-9123-45af-aaee-e8a4bf4b202e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:06.537041 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b15ed-9123-45af-aaee-e8a4bf4b202e service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:06.537328 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b15ed-9123-45af-aaee-e8a4bf4b202e service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:06.537606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b15ed-9123-45af-aaee-e8a4bf4b202e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:06.541871 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a19585-abf7-4744-8833-f69f9292cfa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f77dacc5ddd542128bc051e043b98632, user_id=36c0f6cd473448058186f5a7e2d6ee17, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:06.544841 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a19585-abf7-4744-8833-f69f9292cfa1 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:06.570895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b15ed-9123-45af-aaee-e8a4bf4b202e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:06.577922 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b15ed-9123-45af-aaee-e8a4bf4b202e service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:06.595766 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1031/2038] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:06 2022] PUT /identity/v3/projects/f77dacc5ddd542128bc051e043b98632/users/36c0f6cd473448058186f5a7e2d6ee17/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:06.596688 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1008/2039] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:06.598471 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debcf22-5854-40a9-ac4f-b9d0c46b5a77 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:06.598778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debcf22-5854-40a9-ac4f-b9d0c46b5a77 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:06.599093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debcf22-5854-40a9-ac4f-b9d0c46b5a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:06.666970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-2debcf22-5854-40a9-ac4f-b9d0c46b5a77 None None] MFA Rules not processed for user `36c0f6cd473448058186f5a7e2d6ee17`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:06.668481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2debcf22-5854-40a9-ac4f-b9d0c46b5a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:06.740502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b15ed-9123-45af-aaee-e8a4bf4b202e service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:06.742397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-0c7b15ed-9123-45af-aaee-e8a4bf4b202e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:06.805459 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b15ed-9123-45af-aaee-e8a4bf4b202e service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:06.841997 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:06.921506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1032/2040] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:06.924726 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b713d-6f6f-408e-9eee-76730b1f25df None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:06.926373 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b713d-6f6f-408e-9eee-76730b1f25df None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:06.927154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b713d-6f6f-408e-9eee-76730b1f25df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:06.967471 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-468b713d-6f6f-408e-9eee-76730b1f25df None None] MFA Rules not processed for user `364c6c6f607045d988dd55d8f467b027`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:06.985112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-468b713d-6f6f-408e-9eee-76730b1f25df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:06.995842 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d961691-f526-4753-8c1f-00a0edfd6a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:06.999102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d961691-f526-4753-8c1f-00a0edfd6a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:06.999401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d961691-f526-4753-8c1f-00a0edfd6a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:06.999678 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d961691-f526-4753-8c1f-00a0edfd6a28 admin admin] PATH_INFO: `/v3/projects/b5557a813d8f41f9be8eac38ebf7886e` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:07.008953 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d961691-f526-4753-8c1f-00a0edfd6a28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5557a813d8f41f9be8eac38ebf7886e)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:07.012546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d961691-f526-4753-8c1f-00a0edfd6a28 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:07.042420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:07.112237 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1033/2041] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:07.113431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b713d-6f6f-408e-9eee-76730b1f25df None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:07.116323 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-468b713d-6f6f-408e-9eee-76730b1f25df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:07.138784 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2d961691-f526-4753-8c1f-00a0edfd6a28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5557a813d8f41f9be8eac38ebf7886e', 'request_id': 'req-2d961691-f526-4753-8c1f-00a0edfd6a28'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.159116 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2d961691-f526-4753-8c1f-00a0edfd6a28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5557a813d8f41f9be8eac38ebf7886e', 'request_id': 'req-2d961691-f526-4753-8c1f-00a0edfd6a28'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.213520 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2d961691-f526-4753-8c1f-00a0edfd6a28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5557a813d8f41f9be8eac38ebf7886e', 'request_id': 'req-2d961691-f526-4753-8c1f-00a0edfd6a28'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.213941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b713d-6f6f-408e-9eee-76730b1f25df None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:07.253800 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2d961691-f526-4753-8c1f-00a0edfd6a28 admin admin] The token cache is being invalidate because project b5557a813d8f41f9be8eac38ebf7886e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:07.254190 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2d961691-f526-4753-8c1f-00a0edfd6a28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.255090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2d961691-f526-4753-8c1f-00a0edfd6a28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.257371 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1009/2042] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:06 2022] DELETE /identity/v3/projects/b5557a813d8f41f9be8eac38ebf7886e => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:07.258475 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d961691-f526-4753-8c1f-00a0edfd6a28 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:07.260968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-2d961691-f526-4753-8c1f-00a0edfd6a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:07.290420 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d10fb2-9e86-4967-ac64-b2732f5aa9b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:07.291180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d10fb2-9e86-4967-ac64-b2732f5aa9b9 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:07.291696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d10fb2-9e86-4967-ac64-b2732f5aa9b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:07.291819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d10fb2-9e86-4967-ac64-b2732f5aa9b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:07.293974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-80d10fb2-9e86-4967-ac64-b2732f5aa9b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:07.373031 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d961691-f526-4753-8c1f-00a0edfd6a28 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:07.374756 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:07.378490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d10fb2-9e86-4967-ac64-b2732f5aa9b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:07.380697 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d10fb2-9e86-4967-ac64-b2732f5aa9b9 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:07.384689 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97feacf-9756-4236-b5e9-7cd35efe2bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:07.386098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97feacf-9756-4236-b5e9-7cd35efe2bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:07.386804 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97feacf-9756-4236-b5e9-7cd35efe2bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:07.387093 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97feacf-9756-4236-b5e9-7cd35efe2bcd admin admin] PATH_INFO: `/v3/projects/68aae743fe194a089ea9f28330065b39` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:07.390354 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97feacf-9756-4236-b5e9-7cd35efe2bcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=68aae743fe194a089ea9f28330065b39)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:07.393487 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97feacf-9756-4236-b5e9-7cd35efe2bcd admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:07.396877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1034/2043] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:07 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:07.398117 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d10fb2-9e86-4967-ac64-b2732f5aa9b9 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:07.399882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-80d10fb2-9e86-4967-ac64-b2732f5aa9b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:07.420954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d10fb2-9e86-4967-ac64-b2732f5aa9b9 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:07.430907 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d874c92-c2f8-4b36-a919-006d146a6a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:07.431807 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d874c92-c2f8-4b36-a919-006d146a6a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:07.432673 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d874c92-c2f8-4b36-a919-006d146a6a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:07.433107 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d874c92-c2f8-4b36-a919-006d146a6a73 admin admin] PATH_INFO: `/v3/users/55193f07f44f4d5082c833f661db779f` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:07.434746 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d874c92-c2f8-4b36-a919-006d146a6a73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55193f07f44f4d5082c833f661db779f)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:07.437906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d874c92-c2f8-4b36-a919-006d146a6a73 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:07.486307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f97feacf-9756-4236-b5e9-7cd35efe2bcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68aae743fe194a089ea9f28330065b39', 'request_id': 'req-f97feacf-9756-4236-b5e9-7cd35efe2bcd'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.494546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f97feacf-9756-4236-b5e9-7cd35efe2bcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68aae743fe194a089ea9f28330065b39', 'request_id': 'req-f97feacf-9756-4236-b5e9-7cd35efe2bcd'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.500665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f97feacf-9756-4236-b5e9-7cd35efe2bcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68aae743fe194a089ea9f28330065b39', 'request_id': 'req-f97feacf-9756-4236-b5e9-7cd35efe2bcd'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.519081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f97feacf-9756-4236-b5e9-7cd35efe2bcd admin admin] The token cache is being invalidate because project 68aae743fe194a089ea9f28330065b39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:07.519437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f97feacf-9756-4236-b5e9-7cd35efe2bcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.520464 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9d874c92-c2f8-4b36-a919-006d146a6a73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55193f07f44f4d5082c833f661db779f'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.520711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f97feacf-9756-4236-b5e9-7cd35efe2bcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.527235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1010/2044] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:07 2022] DELETE /identity/v3/projects/68aae743fe194a089ea9f28330065b39 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:07.527235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97feacf-9756-4236-b5e9-7cd35efe2bcd admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:07.528310 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f97feacf-9756-4236-b5e9-7cd35efe2bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:07.536102 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9d874c92-c2f8-4b36-a919-006d146a6a73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55193f07f44f4d5082c833f661db779f'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.542843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9d874c92-c2f8-4b36-a919-006d146a6a73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55193f07f44f4d5082c833f661db779f'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.549974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9d874c92-c2f8-4b36-a919-006d146a6a73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55193f07f44f4d5082c833f661db779f'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.561917 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9d874c92-c2f8-4b36-a919-006d146a6a73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55193f07f44f4d5082c833f661db779f'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.572373 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1035/2045] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:07 2022] DELETE /identity/v3/users/55193f07f44f4d5082c833f661db779f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:07.573476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d874c92-c2f8-4b36-a919-006d146a6a73 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:07.575048 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-9d874c92-c2f8-4b36-a919-006d146a6a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:07.593297 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:07.631051 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97feacf-9756-4236-b5e9-7cd35efe2bcd admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:07.714608 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d874c92-c2f8-4b36-a919-006d146a6a73 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:07.726882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75da917-d36d-403a-b666-3793f8b58fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:07.729243 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75da917-d36d-403a-b666-3793f8b58fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:07.730952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75da917-d36d-403a-b666-3793f8b58fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:07.732060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75da917-d36d-403a-b666-3793f8b58fcb admin admin] PATH_INFO: `/v3/users/0071c867a4fe498780efb7bcd774e131` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:07.734248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a19521-7bcf-43a5-9933-cd94ac38b3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:07.736816 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a19521-7bcf-43a5-9933-cd94ac38b3ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:07.737513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a19521-7bcf-43a5-9933-cd94ac38b3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:07.738050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75da917-d36d-403a-b666-3793f8b58fcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0071c867a4fe498780efb7bcd774e131)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:07.738400 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a19521-7bcf-43a5-9933-cd94ac38b3ee admin admin] PATH_INFO: `/v3/users/cfed562a94774067befe9a30b0e1b947` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:07.740774 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a19521-7bcf-43a5-9933-cd94ac38b3ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfed562a94774067befe9a30b0e1b947)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:07.742652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75da917-d36d-403a-b666-3793f8b58fcb admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:07.743421 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a19521-7bcf-43a5-9933-cd94ac38b3ee admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:07.850519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b75da917-d36d-403a-b666-3793f8b58fcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0071c867a4fe498780efb7bcd774e131'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.859138 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a0a19521-7bcf-43a5-9933-cd94ac38b3ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfed562a94774067befe9a30b0e1b947'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.870063 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b75da917-d36d-403a-b666-3793f8b58fcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0071c867a4fe498780efb7bcd774e131'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.871079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b75da917-d36d-403a-b666-3793f8b58fcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0071c867a4fe498780efb7bcd774e131'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.875906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:07.881256 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b75da917-d36d-403a-b666-3793f8b58fcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0071c867a4fe498780efb7bcd774e131'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.893224 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b75da917-d36d-403a-b666-3793f8b58fcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0071c867a4fe498780efb7bcd774e131'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.896441 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a0a19521-7bcf-43a5-9933-cd94ac38b3ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfed562a94774067befe9a30b0e1b947'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.905311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a0a19521-7bcf-43a5-9933-cd94ac38b3ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfed562a94774067befe9a30b0e1b947'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.905959 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a0a19521-7bcf-43a5-9933-cd94ac38b3ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfed562a94774067befe9a30b0e1b947'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.906700 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a0a19521-7bcf-43a5-9933-cd94ac38b3ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfed562a94774067befe9a30b0e1b947'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:07.911415 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1036/2046] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:07 2022] DELETE /identity/v3/users/0071c867a4fe498780efb7bcd774e131 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:07.936721 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1011/2047] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:07 2022] DELETE /identity/v3/users/cfed562a94774067befe9a30b0e1b947 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:07.942195 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cafc2f-871b-472e-ada1-edb506be6255 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:07.942575 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cafc2f-871b-472e-ada1-edb506be6255 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:07.943975 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cafc2f-871b-472e-ada1-edb506be6255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:07.977149 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-04cafc2f-871b-472e-ada1-edb506be6255 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:07.977293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a19521-7bcf-43a5-9933-cd94ac38b3ee admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:07.978914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-04cafc2f-871b-472e-ada1-edb506be6255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:07.979587 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a0a19521-7bcf-43a5-9933-cd94ac38b3ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:08.102361 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a19521-7bcf-43a5-9933-cd94ac38b3ee admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:08.128240 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ddd7c-ccbf-4a7b-a5c3-98e2f2b62e30 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c0f6cd473448058186f5a7e2d6ee17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f77dacc5ddd542128bc051e043b98632', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:08.131923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ddd7c-ccbf-4a7b-a5c3-98e2f2b62e30 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:08.133008 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ddd7c-ccbf-4a7b-a5c3-98e2f2b62e30 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:08.134056 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ddd7c-ccbf-4a7b-a5c3-98e2f2b62e30 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:08.135222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1037/2048] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:08.135355 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ddd7c-ccbf-4a7b-a5c3-98e2f2b62e30 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:08.137185 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ddd7c-ccbf-4a7b-a5c3-98e2f2b62e30 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:08.143052 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ddd7c-ccbf-4a7b-a5c3-98e2f2b62e30 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:08.151639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:08.155740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cafc2f-871b-472e-ada1-edb506be6255 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:08.158939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-04cafc2f-871b-472e-ada1-edb506be6255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:08.200094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1012/2049] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:49:08.202892 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ddd7c-ccbf-4a7b-a5c3-98e2f2b62e30 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:08.233478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ddd7c-ccbf-4a7b-a5c3-98e2f2b62e30 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:08.247566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51ae41-af23-4d64-8910-c0fabca66d21 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c0f6cd473448058186f5a7e2d6ee17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f77dacc5ddd542128bc051e043b98632', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:08.247995 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51ae41-af23-4d64-8910-c0fabca66d21 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:08.248321 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51ae41-af23-4d64-8910-c0fabca66d21 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:08.248619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51ae41-af23-4d64-8910-c0fabca66d21 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:08.256224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a51ae41-af23-4d64-8910-c0fabca66d21 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:08.258968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a51ae41-af23-4d64-8910-c0fabca66d21 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:08.284510 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cafc2f-871b-472e-ada1-edb506be6255 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:08.293412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91f814-66f1-45ff-95fc-7991b7f40e0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:08.294392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91f814-66f1-45ff-95fc-7991b7f40e0d service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:08.295553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91f814-66f1-45ff-95fc-7991b7f40e0d service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:08.295553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91f814-66f1-45ff-95fc-7991b7f40e0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:08.308380 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91f814-66f1-45ff-95fc-7991b7f40e0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:08.309923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91f814-66f1-45ff-95fc-7991b7f40e0d service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:08.314842 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1013/2050] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:08 2022] POST /identity/v3/projects => generated 355 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:08.321560 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51ae41-af23-4d64-8910-c0fabca66d21 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:08.328986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1038/2051] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:49:08.439130 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51ae41-af23-4d64-8910-c0fabca66d21 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:08.446074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366c9a6-e680-43a3-8de9-9901ad8388be tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c0f6cd473448058186f5a7e2d6ee17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f77dacc5ddd542128bc051e043b98632', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:08.446314 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366c9a6-e680-43a3-8de9-9901ad8388be tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:08.446615 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366c9a6-e680-43a3-8de9-9901ad8388be tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:08.446896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366c9a6-e680-43a3-8de9-9901ad8388be tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:08.447732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366c9a6-e680-43a3-8de9-9901ad8388be tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:08.448050 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366c9a6-e680-43a3-8de9-9901ad8388be tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:08.450425 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366c9a6-e680-43a3-8de9-9901ad8388be tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:08.484809 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1014/2052] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:49:08.503910 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91f814-66f1-45ff-95fc-7991b7f40e0d service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:08.506108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-dd91f814-66f1-45ff-95fc-7991b7f40e0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:08.522212 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91f814-66f1-45ff-95fc-7991b7f40e0d service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:08.597537 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366c9a6-e680-43a3-8de9-9901ad8388be tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:08.616563 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366c9a6-e680-43a3-8de9-9901ad8388be tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:08.630742 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10801eeb-8b31-4d61-a2c4-881a210bb041 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c0f6cd473448058186f5a7e2d6ee17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f77dacc5ddd542128bc051e043b98632', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:08.631686 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10801eeb-8b31-4d61-a2c4-881a210bb041 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:08.643561 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10801eeb-8b31-4d61-a2c4-881a210bb041 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:08.644865 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10801eeb-8b31-4d61-a2c4-881a210bb041 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] PATH_INFO: `/v3/projects/237bbda7a2a347ee8e50ab9e5f3a0e34` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:08.664008 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10801eeb-8b31-4d61-a2c4-881a210bb041 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] RBAC: Authorizing `identity:delete_project(project_id=237bbda7a2a347ee8e50ab9e5f3a0e34)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:08.683907 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10801eeb-8b31-4d61-a2c4-881a210bb041 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:08.709277 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22db75f-f565-4fb2-925e-812129c291e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:08.710205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22db75f-f565-4fb2-925e-812129c291e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:08.710494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22db75f-f565-4fb2-925e-812129c291e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:08.710758 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22db75f-f565-4fb2-925e-812129c291e8 admin admin] PATH_INFO: `/v3/projects/393a6c2119db45b59d82fba236dd420f` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:08.714592 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22db75f-f565-4fb2-925e-812129c291e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=393a6c2119db45b59d82fba236dd420f)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:08.717685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22db75f-f565-4fb2-925e-812129c291e8 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:08.797238 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:08.832732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-10801eeb-8b31-4d61-a2c4-881a210bb041 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '237bbda7a2a347ee8e50ab9e5f3a0e34', 'request_id': 'req-10801eeb-8b31-4d61-a2c4-881a210bb041'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:08.842304 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-10801eeb-8b31-4d61-a2c4-881a210bb041 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '237bbda7a2a347ee8e50ab9e5f3a0e34', 'request_id': 'req-10801eeb-8b31-4d61-a2c4-881a210bb041'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:08.851849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-10801eeb-8b31-4d61-a2c4-881a210bb041 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '237bbda7a2a347ee8e50ab9e5f3a0e34', 'request_id': 'req-10801eeb-8b31-4d61-a2c4-881a210bb041'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:08.868608 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-10801eeb-8b31-4d61-a2c4-881a210bb041 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] The token cache is being invalidate because project 237bbda7a2a347ee8e50ab9e5f3a0e34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:08.868937 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-10801eeb-8b31-4d61-a2c4-881a210bb041 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:08.869464 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-d22db75f-f565-4fb2-925e-812129c291e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '393a6c2119db45b59d82fba236dd420f', 'request_id': 'req-d22db75f-f565-4fb2-925e-812129c291e8'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:08.870539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-10801eeb-8b31-4d61-a2c4-881a210bb041 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:08.876931 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1015/2053] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:08 2022] DELETE /identity/v3/projects/237bbda7a2a347ee8e50ab9e5f3a0e34 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:08.907567 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-d22db75f-f565-4fb2-925e-812129c291e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '393a6c2119db45b59d82fba236dd420f', 'request_id': 'req-d22db75f-f565-4fb2-925e-812129c291e8'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:08.935170 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-d22db75f-f565-4fb2-925e-812129c291e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '393a6c2119db45b59d82fba236dd420f', 'request_id': 'req-d22db75f-f565-4fb2-925e-812129c291e8'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:08.954002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10801eeb-8b31-4d61-a2c4-881a210bb041 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:08.957747 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-10801eeb-8b31-4d61-a2c4-881a210bb041 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:08.959255 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-d22db75f-f565-4fb2-925e-812129c291e8 admin admin] The token cache is being invalidate because project 393a6c2119db45b59d82fba236dd420f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:08.959558 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-d22db75f-f565-4fb2-925e-812129c291e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:08.960324 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-d22db75f-f565-4fb2-925e-812129c291e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:08.962218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1039/2054] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:08 2022] DELETE /identity/v3/projects/393a6c2119db45b59d82fba236dd420f => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:08.973560 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f27617-50c2-4618-bf72-8681fff0b05f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:08.976324 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f27617-50c2-4618-bf72-8681fff0b05f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:08.976647 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f27617-50c2-4618-bf72-8681fff0b05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:09.043566 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-c3f27617-50c2-4618-bf72-8681fff0b05f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:09.044865 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-c3f27617-50c2-4618-bf72-8681fff0b05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:09.091339 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:09.127336 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10801eeb-8b31-4d61-a2c4-881a210bb041 tempest-VolumeQuotaClassesTest-502781031 tempest-VolumeQuotaClassesTest-502781031-project-admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:09.162998 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1040/2055] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:09.164284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f27617-50c2-4618-bf72-8681fff0b05f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:09.166052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-c3f27617-50c2-4618-bf72-8681fff0b05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:09.184392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f27617-50c2-4618-bf72-8681fff0b05f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:09.212676 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a5cc1-40a9-4264-bc21-f53a12874c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:09.214560 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a5cc1-40a9-4264-bc21-f53a12874c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:09.216430 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a5cc1-40a9-4264-bc21-f53a12874c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:09.216938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a5cc1-40a9-4264-bc21-f53a12874c39 admin admin] PATH_INFO: `/v3/projects/b353b54cb4dc4b78bf02e2207f30be3d` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:09.219948 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a5cc1-40a9-4264-bc21-f53a12874c39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b353b54cb4dc4b78bf02e2207f30be3d)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:09.225435 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a5cc1-40a9-4264-bc21-f53a12874c39 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:09.269395 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3eeef3-abca-489b-a415-5f88b5d95d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:09.270729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3eeef3-abca-489b-a415-5f88b5d95d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:09.271061 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3eeef3-abca-489b-a415-5f88b5d95d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:09.272404 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3eeef3-abca-489b-a415-5f88b5d95d32 admin admin] PATH_INFO: `/v3/projects/0da47562c4054af586810c4f47883ccc` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:09.274136 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3eeef3-abca-489b-a415-5f88b5d95d32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0da47562c4054af586810c4f47883ccc)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:09.276472 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3eeef3-abca-489b-a415-5f88b5d95d32 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:09.335271 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-1d9a5cc1-40a9-4264-bc21-f53a12874c39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b353b54cb4dc4b78bf02e2207f30be3d', 'request_id': 'req-1d9a5cc1-40a9-4264-bc21-f53a12874c39'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.343616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-1d9a5cc1-40a9-4264-bc21-f53a12874c39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b353b54cb4dc4b78bf02e2207f30be3d', 'request_id': 'req-1d9a5cc1-40a9-4264-bc21-f53a12874c39'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.358357 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-1d9a5cc1-40a9-4264-bc21-f53a12874c39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b353b54cb4dc4b78bf02e2207f30be3d', 'request_id': 'req-1d9a5cc1-40a9-4264-bc21-f53a12874c39'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.361182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-1a3eeef3-abca-489b-a415-5f88b5d95d32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0da47562c4054af586810c4f47883ccc', 'request_id': 'req-1a3eeef3-abca-489b-a415-5f88b5d95d32'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.378640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-1a3eeef3-abca-489b-a415-5f88b5d95d32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0da47562c4054af586810c4f47883ccc', 'request_id': 'req-1a3eeef3-abca-489b-a415-5f88b5d95d32'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.392992 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-1a3eeef3-abca-489b-a415-5f88b5d95d32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0da47562c4054af586810c4f47883ccc', 'request_id': 'req-1a3eeef3-abca-489b-a415-5f88b5d95d32'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.404026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-1d9a5cc1-40a9-4264-bc21-f53a12874c39 admin admin] The token cache is being invalidate because project b353b54cb4dc4b78bf02e2207f30be3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:09.404575 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-1d9a5cc1-40a9-4264-bc21-f53a12874c39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.405135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-1d9a5cc1-40a9-4264-bc21-f53a12874c39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.406849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1041/2056] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:09 2022] DELETE /identity/v3/projects/b353b54cb4dc4b78bf02e2207f30be3d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:09.408422 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7acac3-1877-414a-a13c-ebe14475086c None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:09.408694 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7acac3-1877-414a-a13c-ebe14475086c None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:09.409011 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7acac3-1877-414a-a13c-ebe14475086c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:09.412560 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-1a3eeef3-abca-489b-a415-5f88b5d95d32 admin admin] The token cache is being invalidate because project 0da47562c4054af586810c4f47883ccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:09.413092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-1a3eeef3-abca-489b-a415-5f88b5d95d32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.414387 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-1a3eeef3-abca-489b-a415-5f88b5d95d32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.416867 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1016/2057] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:08 2022] DELETE /identity/v3/projects/0da47562c4054af586810c4f47883ccc => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:49:09.417953 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3eeef3-abca-489b-a415-5f88b5d95d32 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:09.419682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1a3eeef3-abca-489b-a415-5f88b5d95d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:09.453625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-1d7acac3-1877-414a-a13c-ebe14475086c None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:09.460412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1d7acac3-1877-414a-a13c-ebe14475086c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:09.545476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:09.564010 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3eeef3-abca-489b-a415-5f88b5d95d32 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:09.571396 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8dec7-2bf8-4759-a593-3929eae934f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:09.572411 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8dec7-2bf8-4759-a593-3929eae934f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:09.572714 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8dec7-2bf8-4759-a593-3929eae934f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:09.572999 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8dec7-2bf8-4759-a593-3929eae934f3 admin admin] PATH_INFO: `/v3/users/0219e51f2f404f5e86a1e95400309117` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:09.576633 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8dec7-2bf8-4759-a593-3929eae934f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0219e51f2f404f5e86a1e95400309117)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:09.578732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8dec7-2bf8-4759-a593-3929eae934f3 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:09.583930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1042/2058] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:09.588883 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7acac3-1877-414a-a13c-ebe14475086c None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:09.590780 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1d7acac3-1877-414a-a13c-ebe14475086c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:09.604604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7acac3-1877-414a-a13c-ebe14475086c None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:09.611899 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a89c6-57a0-4949-b87c-b662a10070df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:09.614080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a89c6-57a0-4949-b87c-b662a10070df admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:09.615047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a89c6-57a0-4949-b87c-b662a10070df admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:09.615427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a89c6-57a0-4949-b87c-b662a10070df admin admin] PATH_INFO: `/v3/users/9469b989d0b84d5897106762a5e77c7d` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:09.617530 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a89c6-57a0-4949-b87c-b662a10070df admin admin] RBAC: Authorizing `identity:delete_user(user_id=9469b989d0b84d5897106762a5e77c7d)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:09.621399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a89c6-57a0-4949-b87c-b662a10070df admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:09.637119 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:09.671807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4ab8dec7-2bf8-4759-a593-3929eae934f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0219e51f2f404f5e86a1e95400309117'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.688086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4ab8dec7-2bf8-4759-a593-3929eae934f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0219e51f2f404f5e86a1e95400309117'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.693353 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-cb2a89c6-57a0-4949-b87c-b662a10070df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9469b989d0b84d5897106762a5e77c7d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.700331 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4ab8dec7-2bf8-4759-a593-3929eae934f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0219e51f2f404f5e86a1e95400309117'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.701355 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4ab8dec7-2bf8-4759-a593-3929eae934f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0219e51f2f404f5e86a1e95400309117'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.702086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4ab8dec7-2bf8-4759-a593-3929eae934f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0219e51f2f404f5e86a1e95400309117'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.710818 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-cb2a89c6-57a0-4949-b87c-b662a10070df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9469b989d0b84d5897106762a5e77c7d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.711579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-cb2a89c6-57a0-4949-b87c-b662a10070df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9469b989d0b84d5897106762a5e77c7d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.720395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-cb2a89c6-57a0-4949-b87c-b662a10070df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9469b989d0b84d5897106762a5e77c7d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.727349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1017/2059] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:09 2022] DELETE /identity/v3/users/0219e51f2f404f5e86a1e95400309117 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:09.729362 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5bc81-0784-41e4-8ac5-1451918f0112 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:09.730130 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5bc81-0784-41e4-8ac5-1451918f0112 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:09.730363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-cb2a89c6-57a0-4949-b87c-b662a10070df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9469b989d0b84d5897106762a5e77c7d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:09.730771 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5bc81-0784-41e4-8ac5-1451918f0112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:09.742918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1043/2060] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:09 2022] DELETE /identity/v3/users/9469b989d0b84d5897106762a5e77c7d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:09.744088 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a89c6-57a0-4949-b87c-b662a10070df admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:09.745459 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-cb2a89c6-57a0-4949-b87c-b662a10070df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:09.768847 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-18c5bc81-0784-41e4-8ac5-1451918f0112 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:09.775096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-18c5bc81-0784-41e4-8ac5-1451918f0112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:09.891270 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a89c6-57a0-4949-b87c-b662a10070df admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:09.904521 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e072330-3505-46ca-b110-eaf6e577da26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:09.905447 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e072330-3505-46ca-b110-eaf6e577da26 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:09.905864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e072330-3505-46ca-b110-eaf6e577da26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:09.906219 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e072330-3505-46ca-b110-eaf6e577da26 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:09.912573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e072330-3505-46ca-b110-eaf6e577da26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:09.913034 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e072330-3505-46ca-b110-eaf6e577da26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:09.916577 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e072330-3505-46ca-b110-eaf6e577da26 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:09.931887 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1018/2061] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:09.933301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b700da-a94c-4df0-a5e7-4bd94141c1c5 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:09.933594 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b700da-a94c-4df0-a5e7-4bd94141c1c5 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:09.933906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b700da-a94c-4df0-a5e7-4bd94141c1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:09.952868 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1044/2062] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:49:09.954353 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e774a5-0721-422f-806a-2591685df380 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:09.954757 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e774a5-0721-422f-806a-2591685df380 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:09.955108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e774a5-0721-422f-806a-2591685df380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:09.989081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-75e774a5-0721-422f-806a-2591685df380 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:09.991111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-a9b700da-a94c-4df0-a5e7-4bd94141c1c5 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:09.992459 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-75e774a5-0721-422f-806a-2591685df380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:09.993426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a9b700da-a94c-4df0-a5e7-4bd94141c1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:10.012157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1019/2063] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:10.020199 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1045/2064] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:10.022657 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e774a5-0721-422f-806a-2591685df380 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:10.024299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b700da-a94c-4df0-a5e7-4bd94141c1c5 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:10.040584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b700da-a94c-4df0-a5e7-4bd94141c1c5 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:10.041487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e774a5-0721-422f-806a-2591685df380 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:10.046764 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde514e3-6c7d-4355-8fad-f7ba4c8894e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:10.047761 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde514e3-6c7d-4355-8fad-f7ba4c8894e3 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:10.048138 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde514e3-6c7d-4355-8fad-f7ba4c8894e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:10.048247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac7485-e611-4310-9f0b-bc417d7dabb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:10.048597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde514e3-6c7d-4355-8fad-f7ba4c8894e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:10.048814 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac7485-e611-4310-9f0b-bc417d7dabb7 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:10.049107 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac7485-e611-4310-9f0b-bc417d7dabb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:10.049387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac7485-e611-4310-9f0b-bc417d7dabb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:10.049739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde514e3-6c7d-4355-8fad-f7ba4c8894e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:10.050163 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac7485-e611-4310-9f0b-bc417d7dabb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:10.052046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac7485-e611-4310-9f0b-bc417d7dabb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:10.052462 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde514e3-6c7d-4355-8fad-f7ba4c8894e3 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:10.055783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac7485-e611-4310-9f0b-bc417d7dabb7 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:10.084452 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1046/2065] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:49:10.085476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac7485-e611-4310-9f0b-bc417d7dabb7 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:10.087314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-b2ac7485-e611-4310-9f0b-bc417d7dabb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:10.102689 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1020/2066] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:10 2022] POST /identity/v3/projects => generated 365 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:10.105477 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde514e3-6c7d-4355-8fad-f7ba4c8894e3 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:10.137483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:10.175773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac7485-e611-4310-9f0b-bc417d7dabb7 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:10.237313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde514e3-6c7d-4355-8fad-f7ba4c8894e3 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:10.237924 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee334dfe-1e8d-4836-836d-b970205e41e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:10.238414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee334dfe-1e8d-4836-836d-b970205e41e5 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:10.238700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee334dfe-1e8d-4836-836d-b970205e41e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:10.238995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee334dfe-1e8d-4836-836d-b970205e41e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:10.247592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3b080-410d-44c3-a540-ec819f2f7a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:10.248693 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3b080-410d-44c3-a540-ec819f2f7a82 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:10.249188 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3b080-410d-44c3-a540-ec819f2f7a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:10.249725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3b080-410d-44c3-a540-ec819f2f7a82 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:10.250661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3b080-410d-44c3-a540-ec819f2f7a82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:10.252940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3b080-410d-44c3-a540-ec819f2f7a82 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:10.259377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee334dfe-1e8d-4836-836d-b970205e41e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:10.260837 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee334dfe-1e8d-4836-836d-b970205e41e5 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:10.279145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1047/2067] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:10.279969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee334dfe-1e8d-4836-836d-b970205e41e5 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:10.290892 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1021/2068] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:10 2022] POST /identity/v3/projects => generated 373 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:10.294875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3b080-410d-44c3-a540-ec819f2f7a82 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:10.296444 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee334dfe-1e8d-4836-836d-b970205e41e5 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:10.304393 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:10.386953 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f7bbe-6877-4990-aebd-25d730681a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:10.388045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f7bbe-6877-4990-aebd-25d730681a65 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:10.388433 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f7bbe-6877-4990-aebd-25d730681a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:10.389035 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f7bbe-6877-4990-aebd-25d730681a65 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:10.391455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f7bbe-6877-4990-aebd-25d730681a65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:10.392801 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f7bbe-6877-4990-aebd-25d730681a65 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:10.403536 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3b080-410d-44c3-a540-ec819f2f7a82 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:10.410660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e548f46-9c4e-47ea-a2a7-18d76a237c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:10.411582 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e548f46-9c4e-47ea-a2a7-18d76a237c1e admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:10.412150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e548f46-9c4e-47ea-a2a7-18d76a237c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:10.412604 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e548f46-9c4e-47ea-a2a7-18d76a237c1e admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:10.413094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e548f46-9c4e-47ea-a2a7-18d76a237c1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:10.417446 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e548f46-9c4e-47ea-a2a7-18d76a237c1e admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:10.426469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1048/2069] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:10 2022] POST /identity/v3/users => generated 283 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:10.431157 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f7bbe-6877-4990-aebd-25d730681a65 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:10.455020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f7bbe-6877-4990-aebd-25d730681a65 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:10.455251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1022/2070] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:10 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:49:10.456440 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e548f46-9c4e-47ea-a2a7-18d76a237c1e admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:10.461078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96d4a7-d8ad-43d4-8e29-fdf1dec00dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:10.461923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96d4a7-d8ad-43d4-8e29-fdf1dec00dfc admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:10.462262 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96d4a7-d8ad-43d4-8e29-fdf1dec00dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:10.462712 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e96d4a7-d8ad-43d4-8e29-fdf1dec00dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:10.463661 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96d4a7-d8ad-43d4-8e29-fdf1dec00dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:10.465599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e96d4a7-d8ad-43d4-8e29-fdf1dec00dfc admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:10.482224 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1049/2071] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:10.484900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e548f46-9c4e-47ea-a2a7-18d76a237c1e admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:10.486506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96d4a7-d8ad-43d4-8e29-fdf1dec00dfc admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:10.490886 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0635254-1b96-415b-8837-64428f53f57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:10.492175 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0635254-1b96-415b-8837-64428f53f57d admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:10.492900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0635254-1b96-415b-8837-64428f53f57d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:10.493185 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0635254-1b96-415b-8837-64428f53f57d admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:10.494926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0635254-1b96-415b-8837-64428f53f57d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:10.497609 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0635254-1b96-415b-8837-64428f53f57d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:10.500066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e96d4a7-d8ad-43d4-8e29-fdf1dec00dfc admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:10.511081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d59742-53fb-40c4-82c4-b12178c9c5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:10.512080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d59742-53fb-40c4-82c4-b12178c9c5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:10.512519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d59742-53fb-40c4-82c4-b12178c9c5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:10.512951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d59742-53fb-40c4-82c4-b12178c9c5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:10.514538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d59742-53fb-40c4-82c4-b12178c9c5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:10.516675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d59742-53fb-40c4-82c4-b12178c9c5d8 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:10.525852 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1023/2072] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:49:10.527315 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0635254-1b96-415b-8837-64428f53f57d admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:10.537861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1050/2073] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:10.541936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0635254-1b96-415b-8837-64428f53f57d admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:10.543938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d59742-53fb-40c4-82c4-b12178c9c5d8 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:10.558660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d59742-53fb-40c4-82c4-b12178c9c5d8 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:10.562468 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c131e3-b0ca-480f-86bf-bf53e05051e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:10.563171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c131e3-b0ca-480f-86bf-bf53e05051e1 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:10.563462 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c131e3-b0ca-480f-86bf-bf53e05051e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:10.563742 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c131e3-b0ca-480f-86bf-bf53e05051e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:10.563973 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0cd7b-0718-4aa3-9a70-c25c0ae16821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:10.564485 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c131e3-b0ca-480f-86bf-bf53e05051e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:10.564861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0cd7b-0718-4aa3-9a70-c25c0ae16821 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:10.565145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0cd7b-0718-4aa3-9a70-c25c0ae16821 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:10.565421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0cd7b-0718-4aa3-9a70-c25c0ae16821 admin admin] PATH_INFO: `/v3/projects/72fff1ebf5e044b09b04b80454c87d57/users/91ad7f548b55489a80c395dfaa0efa35/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:10.566151 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c131e3-b0ca-480f-86bf-bf53e05051e1 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:10.580582 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1024/2074] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:10.585691 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c131e3-b0ca-480f-86bf-bf53e05051e1 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:10.587871 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0cd7b-0718-4aa3-9a70-c25c0ae16821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72fff1ebf5e044b09b04b80454c87d57, user_id=91ad7f548b55489a80c395dfaa0efa35, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:10.591539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0cd7b-0718-4aa3-9a70-c25c0ae16821 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:10.595953 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c131e3-b0ca-480f-86bf-bf53e05051e1 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:10.609127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035cd02-ab3e-4103-a0f6-e581d87cab8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:10.609127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035cd02-ab3e-4103-a0f6-e581d87cab8e admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:10.609127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035cd02-ab3e-4103-a0f6-e581d87cab8e admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:10.609127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035cd02-ab3e-4103-a0f6-e581d87cab8e admin admin] PATH_INFO: `/v3/projects/78b694283b9142afbedb2fd7b41293ec/users/188ff69c7ea54bcba44acdc1218363f8/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:10.618123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1051/2075] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:10 2022] PUT /identity/v3/projects/72fff1ebf5e044b09b04b80454c87d57/users/91ad7f548b55489a80c395dfaa0efa35/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:10.623050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebd942-2177-4833-8b1f-63ec7f97a538 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:10.623454 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebd942-2177-4833-8b1f-63ec7f97a538 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:10.623828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebd942-2177-4833-8b1f-63ec7f97a538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:10.639026 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035cd02-ab3e-4103-a0f6-e581d87cab8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=78b694283b9142afbedb2fd7b41293ec, user_id=188ff69c7ea54bcba44acdc1218363f8, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:10.643116 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035cd02-ab3e-4103-a0f6-e581d87cab8e admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:10.661273 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-80ebd942-2177-4833-8b1f-63ec7f97a538 None None] MFA Rules not processed for user `91ad7f548b55489a80c395dfaa0efa35`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:10.662899 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-80ebd942-2177-4833-8b1f-63ec7f97a538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:10.666086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1025/2076] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:10 2022] PUT /identity/v3/projects/78b694283b9142afbedb2fd7b41293ec/users/188ff69c7ea54bcba44acdc1218363f8/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:10.676372 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a33e4-8316-4630-be32-cacc0aed5842 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:10.677097 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a33e4-8316-4630-be32-cacc0aed5842 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:10.677687 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a33e4-8316-4630-be32-cacc0aed5842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:10.688202 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:10.717022 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-6c1a33e4-8316-4630-be32-cacc0aed5842 None None] MFA Rules not processed for user `188ff69c7ea54bcba44acdc1218363f8`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:10.718269 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-6c1a33e4-8316-4630-be32-cacc0aed5842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:10.763547 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1052/2077] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:10.764720 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebd942-2177-4833-8b1f-63ec7f97a538 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:10.766397 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-80ebd942-2177-4833-8b1f-63ec7f97a538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:10.819426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1026/2078] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:10.823396 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a33e4-8316-4630-be32-cacc0aed5842 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:10.850981 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebd942-2177-4833-8b1f-63ec7f97a538 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:10.856545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de10dea-b72e-48cf-b272-9f0bc725196e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:10.857408 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de10dea-b72e-48cf-b272-9f0bc725196e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:10.857963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de10dea-b72e-48cf-b272-9f0bc725196e admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:10.858073 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de10dea-b72e-48cf-b272-9f0bc725196e admin admin] PATH_INFO: `/v3/projects/b3c579ca29a1409ab0b60a586694729e` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:10.859327 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de10dea-b72e-48cf-b272-9f0bc725196e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3c579ca29a1409ab0b60a586694729e)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:10.861248 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de10dea-b72e-48cf-b272-9f0bc725196e admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:10.879508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:10.910994 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a33e4-8316-4630-be32-cacc0aed5842 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:10.917397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420334ca-86ca-48d3-b427-407b05e1c7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:10.918388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420334ca-86ca-48d3-b427-407b05e1c7af admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:10.918575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420334ca-86ca-48d3-b427-407b05e1c7af admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:10.918925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420334ca-86ca-48d3-b427-407b05e1c7af admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:10.919723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420334ca-86ca-48d3-b427-407b05e1c7af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:10.922273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420334ca-86ca-48d3-b427-407b05e1c7af admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:10.937785 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5de10dea-b72e-48cf-b272-9f0bc725196e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3c579ca29a1409ab0b60a586694729e', 'request_id': 'req-5de10dea-b72e-48cf-b272-9f0bc725196e'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:10.946892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5de10dea-b72e-48cf-b272-9f0bc725196e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3c579ca29a1409ab0b60a586694729e', 'request_id': 'req-5de10dea-b72e-48cf-b272-9f0bc725196e'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:10.953836 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1027/2079] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:10 2022] POST /identity/v3/projects => generated 365 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:10.954968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420334ca-86ca-48d3-b427-407b05e1c7af admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:10.959003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5de10dea-b72e-48cf-b272-9f0bc725196e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3c579ca29a1409ab0b60a586694729e', 'request_id': 'req-5de10dea-b72e-48cf-b272-9f0bc725196e'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:10.971857 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5de10dea-b72e-48cf-b272-9f0bc725196e admin admin] The token cache is being invalidate because project b3c579ca29a1409ab0b60a586694729e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:10.972205 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5de10dea-b72e-48cf-b272-9f0bc725196e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:10.973507 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5de10dea-b72e-48cf-b272-9f0bc725196e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:10.975529 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1053/2080] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:10 2022] DELETE /identity/v3/projects/b3c579ca29a1409ab0b60a586694729e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:10.976458 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de10dea-b72e-48cf-b272-9f0bc725196e admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:10.978036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-5de10dea-b72e-48cf-b272-9f0bc725196e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:11.048656 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420334ca-86ca-48d3-b427-407b05e1c7af admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:11.057142 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92a6a5-1716-4223-b6ec-6cf6cb9ae9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:11.058186 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92a6a5-1716-4223-b6ec-6cf6cb9ae9d3 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:11.059056 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92a6a5-1716-4223-b6ec-6cf6cb9ae9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:11.059588 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92a6a5-1716-4223-b6ec-6cf6cb9ae9d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:11.061261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c92a6a5-1716-4223-b6ec-6cf6cb9ae9d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:11.064140 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c92a6a5-1716-4223-b6ec-6cf6cb9ae9d3 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:11.077169 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:11.098251 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de10dea-b72e-48cf-b272-9f0bc725196e admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:11.105586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1028/2081] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:10 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:11.105586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ba268-2564-44be-9e42-df5412caea50 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:11.105586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ba268-2564-44be-9e42-df5412caea50 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:11.105586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ba268-2564-44be-9e42-df5412caea50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:11.106907 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d0084-5529-4274-a7a6-90f9fb8abf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:11.107629 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d0084-5529-4274-a7a6-90f9fb8abf7c admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:11.107931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d0084-5529-4274-a7a6-90f9fb8abf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:11.108222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d0084-5529-4274-a7a6-90f9fb8abf7c admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:11.108921 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d0084-5529-4274-a7a6-90f9fb8abf7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:11.110906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d0084-5529-4274-a7a6-90f9fb8abf7c admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:11.128495 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-f44ba268-2564-44be-9e42-df5412caea50 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:11.136037 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f44ba268-2564-44be-9e42-df5412caea50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:11.144270 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1054/2082] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:10 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:11.145323 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d0084-5529-4274-a7a6-90f9fb8abf7c admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:11.150056 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-4a0d0084-5529-4274-a7a6-90f9fb8abf7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:11.165766 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d0084-5529-4274-a7a6-90f9fb8abf7c admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:11.203176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1029/2083] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:11.210542 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ba268-2564-44be-9e42-df5412caea50 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:11.211871 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f44ba268-2564-44be-9e42-df5412caea50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:11.225271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ba268-2564-44be-9e42-df5412caea50 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:11.231941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666afba-2201-4a79-9756-9e2f314a1c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:11.232631 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666afba-2201-4a79-9756-9e2f314a1c97 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:11.232914 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666afba-2201-4a79-9756-9e2f314a1c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:11.233195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666afba-2201-4a79-9756-9e2f314a1c97 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:11.233931 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666afba-2201-4a79-9756-9e2f314a1c97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:11.235901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666afba-2201-4a79-9756-9e2f314a1c97 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:11.243874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46896e92-0abf-46ad-8b5e-7395c3003ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:11.244825 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46896e92-0abf-46ad-8b5e-7395c3003ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:11.245188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46896e92-0abf-46ad-8b5e-7395c3003ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:11.245529 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46896e92-0abf-46ad-8b5e-7395c3003ccb admin admin] PATH_INFO: `/v3/projects/7c73b6b08d8d460a8f67e24f6d0e8a2d` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:11.246823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46896e92-0abf-46ad-8b5e-7395c3003ccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c73b6b08d8d460a8f67e24f6d0e8a2d)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:11.248734 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46896e92-0abf-46ad-8b5e-7395c3003ccb admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:11.261222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1030/2084] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:11 2022] POST /identity/v3/users => generated 293 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:11.261222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666afba-2201-4a79-9756-9e2f314a1c97 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:11.269388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666afba-2201-4a79-9756-9e2f314a1c97 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:11.276307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c747a-0cf5-4a7b-a43e-b74f57b9300c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:11.277204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c747a-0cf5-4a7b-a43e-b74f57b9300c admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:11.277536 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c747a-0cf5-4a7b-a43e-b74f57b9300c admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:11.277813 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c747a-0cf5-4a7b-a43e-b74f57b9300c admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:11.278548 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c747a-0cf5-4a7b-a43e-b74f57b9300c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:11.280169 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c747a-0cf5-4a7b-a43e-b74f57b9300c admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:11.294338 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1031/2085] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:11.295841 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c879b-8f7e-4635-b652-bfa6a90a0b53 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:11.296136 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c879b-8f7e-4635-b652-bfa6a90a0b53 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:11.296671 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c879b-8f7e-4635-b652-bfa6a90a0b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:11.304753 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-46896e92-0abf-46ad-8b5e-7395c3003ccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c73b6b08d8d460a8f67e24f6d0e8a2d', 'request_id': 'req-46896e92-0abf-46ad-8b5e-7395c3003ccb'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:11.311056 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-46896e92-0abf-46ad-8b5e-7395c3003ccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c73b6b08d8d460a8f67e24f6d0e8a2d', 'request_id': 'req-46896e92-0abf-46ad-8b5e-7395c3003ccb'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:11.316681 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-e17c879b-8f7e-4635-b652-bfa6a90a0b53 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:11.318749 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e17c879b-8f7e-4635-b652-bfa6a90a0b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:11.321014 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-46896e92-0abf-46ad-8b5e-7395c3003ccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c73b6b08d8d460a8f67e24f6d0e8a2d', 'request_id': 'req-46896e92-0abf-46ad-8b5e-7395c3003ccb'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:11.332686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-46896e92-0abf-46ad-8b5e-7395c3003ccb admin admin] The token cache is being invalidate because project 7c73b6b08d8d460a8f67e24f6d0e8a2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:11.332978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:11.333361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-46896e92-0abf-46ad-8b5e-7395c3003ccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:11.334066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-46896e92-0abf-46ad-8b5e-7395c3003ccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:11.336170 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1055/2086] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:11 2022] DELETE /identity/v3/projects/7c73b6b08d8d460a8f67e24f6d0e8a2d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:11.337430 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46896e92-0abf-46ad-8b5e-7395c3003ccb admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:11.340098 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-46896e92-0abf-46ad-8b5e-7395c3003ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:11.423134 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1032/2087] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:11.423983 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c879b-8f7e-4635-b652-bfa6a90a0b53 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:11.429675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46896e92-0abf-46ad-8b5e-7395c3003ccb admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:11.430068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e17c879b-8f7e-4635-b652-bfa6a90a0b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:11.437739 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235483d-961e-4ce0-8b4f-cf81931a34fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:11.438604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235483d-961e-4ce0-8b4f-cf81931a34fe admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:11.438918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235483d-961e-4ce0-8b4f-cf81931a34fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:11.439196 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235483d-961e-4ce0-8b4f-cf81931a34fe admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:11.440030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235483d-961e-4ce0-8b4f-cf81931a34fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:11.441874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235483d-961e-4ce0-8b4f-cf81931a34fe admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:11.444901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c879b-8f7e-4635-b652-bfa6a90a0b53 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:11.452950 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76038f7-b2c3-4f80-a3a9-e582fb2f059f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:11.454331 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76038f7-b2c3-4f80-a3a9-e582fb2f059f admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:11.455211 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76038f7-b2c3-4f80-a3a9-e582fb2f059f admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:11.455661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76038f7-b2c3-4f80-a3a9-e582fb2f059f admin admin] PATH_INFO: `/v3/projects/971b6e0af6014bf68ad1e6fe76e02380/users/5af4b3ba6af143a496939d1e620a9b3d/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:11.460480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1056/2088] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:11.460480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235483d-961e-4ce0-8b4f-cf81931a34fe admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:11.460480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f235483d-961e-4ce0-8b4f-cf81931a34fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:11.474123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235483d-961e-4ce0-8b4f-cf81931a34fe admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:11.479543 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc300e43-c504-4fe6-961f-7f48952c84a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:11.480531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc300e43-c504-4fe6-961f-7f48952c84a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:11.480918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc300e43-c504-4fe6-961f-7f48952c84a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:11.481293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc300e43-c504-4fe6-961f-7f48952c84a1 admin admin] PATH_INFO: `/v3/users/36c0f6cd473448058186f5a7e2d6ee17` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:11.482622 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc300e43-c504-4fe6-961f-7f48952c84a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36c0f6cd473448058186f5a7e2d6ee17)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:11.483049 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76038f7-b2c3-4f80-a3a9-e582fb2f059f admin admin] RBAC: Authorizing `identity:create_grant(project_id=971b6e0af6014bf68ad1e6fe76e02380, user_id=5af4b3ba6af143a496939d1e620a9b3d, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:11.485067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc300e43-c504-4fe6-961f-7f48952c84a1 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:11.487045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76038f7-b2c3-4f80-a3a9-e582fb2f059f admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:11.488166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:11.504746 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1033/2089] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:11 2022] PUT /identity/v3/projects/971b6e0af6014bf68ad1e6fe76e02380/users/5af4b3ba6af143a496939d1e620a9b3d/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:11.506647 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76038f7-b2c3-4f80-a3a9-e582fb2f059f admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:11.563320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-bc300e43-c504-4fe6-961f-7f48952c84a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36c0f6cd473448058186f5a7e2d6ee17'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:11.579017 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-bc300e43-c504-4fe6-961f-7f48952c84a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36c0f6cd473448058186f5a7e2d6ee17'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:11.579783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-bc300e43-c504-4fe6-961f-7f48952c84a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36c0f6cd473448058186f5a7e2d6ee17'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:11.589656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-bc300e43-c504-4fe6-961f-7f48952c84a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36c0f6cd473448058186f5a7e2d6ee17'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:11.597344 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76038f7-b2c3-4f80-a3a9-e582fb2f059f admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:11.598334 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-bc300e43-c504-4fe6-961f-7f48952c84a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36c0f6cd473448058186f5a7e2d6ee17'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:11.605996 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1057/2090] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:11 2022] DELETE /identity/v3/users/36c0f6cd473448058186f5a7e2d6ee17 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:11.609162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc300e43-c504-4fe6-961f-7f48952c84a1 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:11.609438 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680a661-a28b-4c10-ba84-c3eb775733ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:11.609546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-bc300e43-c504-4fe6-961f-7f48952c84a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:11.609662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680a661-a28b-4c10-ba84-c3eb775733ec admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:11.609662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680a661-a28b-4c10-ba84-c3eb775733ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:11.609662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680a661-a28b-4c10-ba84-c3eb775733ec admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:11.610656 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680a661-a28b-4c10-ba84-c3eb775733ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:11.611360 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680a661-a28b-4c10-ba84-c3eb775733ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:11.613500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680a661-a28b-4c10-ba84-c3eb775733ec admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:11.631921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1034/2091] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:49:11.632954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680a661-a28b-4c10-ba84-c3eb775733ec admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:11.635386 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-8680a661-a28b-4c10-ba84-c3eb775733ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:11.666528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680a661-a28b-4c10-ba84-c3eb775733ec admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:11.685938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc300e43-c504-4fe6-961f-7f48952c84a1 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:11.691227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f48521-fe17-40f1-81be-487462c4ec6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:11.692099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f48521-fe17-40f1-81be-487462c4ec6b admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:11.692506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f48521-fe17-40f1-81be-487462c4ec6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:11.693345 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f48521-fe17-40f1-81be-487462c4ec6b admin admin] PATH_INFO: `/v3/projects/442ec400ec684463ac80c68137d33b67/users/9de4b7be2ed34036868c692ab091b3c8/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:11.710915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f48521-fe17-40f1-81be-487462c4ec6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=442ec400ec684463ac80c68137d33b67, user_id=9de4b7be2ed34036868c692ab091b3c8, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:11.713928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f48521-fe17-40f1-81be-487462c4ec6b admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:11.734717 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1058/2092] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:11 2022] PUT /identity/v3/projects/442ec400ec684463ac80c68137d33b67/users/9de4b7be2ed34036868c692ab091b3c8/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:11.735777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f48521-fe17-40f1-81be-487462c4ec6b admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:11.738174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-81f48521-fe17-40f1-81be-487462c4ec6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:11.747518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54082c1a-a4ca-48d1-85cc-3a5b7ff8edde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:11.748480 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54082c1a-a4ca-48d1-85cc-3a5b7ff8edde admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:11.748678 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54082c1a-a4ca-48d1-85cc-3a5b7ff8edde admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:11.749755 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54082c1a-a4ca-48d1-85cc-3a5b7ff8edde admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:11.749890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54082c1a-a4ca-48d1-85cc-3a5b7ff8edde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:11.751476 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54082c1a-a4ca-48d1-85cc-3a5b7ff8edde admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:11.766160 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1035/2093] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:11.766160 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54082c1a-a4ca-48d1-85cc-3a5b7ff8edde admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:11.779134 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54082c1a-a4ca-48d1-85cc-3a5b7ff8edde admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:11.807189 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:11.847105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d0a66-e744-43ce-af99-2444585bce2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:11.847786 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d0a66-e744-43ce-af99-2444585bce2b admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:11.848070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d0a66-e744-43ce-af99-2444585bce2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:11.848537 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f48521-fe17-40f1-81be-487462c4ec6b admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:11.849317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d0a66-e744-43ce-af99-2444585bce2b admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:11.850190 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511d0a66-e744-43ce-af99-2444585bce2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:11.852223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511d0a66-e744-43ce-af99-2444585bce2b admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:11.857000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e80ed-5845-4b1b-b5f3-04d5a812d425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:11.857710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e80ed-5845-4b1b-b5f3-04d5a812d425 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:11.857997 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e80ed-5845-4b1b-b5f3-04d5a812d425 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:11.858270 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e80ed-5845-4b1b-b5f3-04d5a812d425 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:11.858968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175e80ed-5845-4b1b-b5f3-04d5a812d425 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:11.860889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175e80ed-5845-4b1b-b5f3-04d5a812d425 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:11.863953 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1036/2094] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:11.869315 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d0a66-e744-43ce-af99-2444585bce2b admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:11.871049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:11.882419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d0a66-e744-43ce-af99-2444585bce2b admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:11.888315 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234a286-dc52-421c-b667-39168839615f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:11.889127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234a286-dc52-421c-b667-39168839615f admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:11.889408 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234a286-dc52-421c-b667-39168839615f admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:11.889712 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234a286-dc52-421c-b667-39168839615f admin admin] PATH_INFO: `/v3/projects/971b6e0af6014bf68ad1e6fe76e02380/users/5af4b3ba6af143a496939d1e620a9b3d/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:11.891772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234a286-dc52-421c-b667-39168839615f admin admin] RBAC: Authorizing `identity:create_grant(project_id=971b6e0af6014bf68ad1e6fe76e02380, user_id=5af4b3ba6af143a496939d1e620a9b3d, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:11.894686 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234a286-dc52-421c-b667-39168839615f admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:11.902344 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1059/2095] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:11 2022] POST /identity/v3/projects => generated 369 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:11.905257 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e80ed-5845-4b1b-b5f3-04d5a812d425 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:11.916165 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1037/2096] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:11 2022] PUT /identity/v3/projects/971b6e0af6014bf68ad1e6fe76e02380/users/5af4b3ba6af143a496939d1e620a9b3d/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:11.917028 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234a286-dc52-421c-b667-39168839615f admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:11.994638 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234a286-dc52-421c-b667-39168839615f admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:12.001172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd53e90-0dab-40c1-804d-4c21b165e903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:12.002288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd53e90-0dab-40c1-804d-4c21b165e903 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:12.002768 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd53e90-0dab-40c1-804d-4c21b165e903 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:12.003760 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd53e90-0dab-40c1-804d-4c21b165e903 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:12.004713 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd53e90-0dab-40c1-804d-4c21b165e903 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:12.007635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd53e90-0dab-40c1-804d-4c21b165e903 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:12.041389 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1038/2097] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:11 2022] POST /identity/v3/users => generated 285 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:12.043341 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447b35b-acf0-4237-a624-f1796f56dde7 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:12.043789 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447b35b-acf0-4237-a624-f1796f56dde7 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:12.044240 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447b35b-acf0-4237-a624-f1796f56dde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:12.061197 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e80ed-5845-4b1b-b5f3-04d5a812d425 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:12.066121 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43eb0b-ecbb-4cdd-a889-d10abc00dc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:12.067023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43eb0b-ecbb-4cdd-a889-d10abc00dc38 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:12.067307 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43eb0b-ecbb-4cdd-a889-d10abc00dc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:12.068122 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43eb0b-ecbb-4cdd-a889-d10abc00dc38 admin admin] PATH_INFO: `/v3/projects/442ec400ec684463ac80c68137d33b67/users/9de4b7be2ed34036868c692ab091b3c8/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:12.070250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43eb0b-ecbb-4cdd-a889-d10abc00dc38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=442ec400ec684463ac80c68137d33b67, user_id=9de4b7be2ed34036868c692ab091b3c8, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:12.074118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43eb0b-ecbb-4cdd-a889-d10abc00dc38 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:12.094425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1060/2098] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:11 2022] PUT /identity/v3/projects/442ec400ec684463ac80c68137d33b67/users/9de4b7be2ed34036868c692ab091b3c8/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:12.100247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43eb0b-ecbb-4cdd-a889-d10abc00dc38 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:12.112059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-c447b35b-acf0-4237-a624-f1796f56dde7 None None] MFA Rules not processed for user `5af4b3ba6af143a496939d1e620a9b3d`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:12.113296 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-c447b35b-acf0-4237-a624-f1796f56dde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:12.241220 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43eb0b-ecbb-4cdd-a889-d10abc00dc38 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:12.259327 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1039/2099] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:12 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:12.259490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c8376-2ed4-4151-b3ef-d4fbb994bf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:12.260904 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799b936-c8cf-4321-b0db-3c59621772af None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:12.261102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799b936-c8cf-4321-b0db-3c59621772af None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:12.261387 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799b936-c8cf-4321-b0db-3c59621772af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:12.262378 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c8376-2ed4-4151-b3ef-d4fbb994bf4e admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:12.262770 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c8376-2ed4-4151-b3ef-d4fbb994bf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:12.263809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c8376-2ed4-4151-b3ef-d4fbb994bf4e admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:12.276203 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c8376-2ed4-4151-b3ef-d4fbb994bf4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:12.284376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c8376-2ed4-4151-b3ef-d4fbb994bf4e admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:12.299622 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1061/2100] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:12.300591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c8376-2ed4-4151-b3ef-d4fbb994bf4e admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:12.303948 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-232c8376-2ed4-4151-b3ef-d4fbb994bf4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:12.321769 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-f799b936-c8cf-4321-b0db-3c59621772af None None] MFA Rules not processed for user `9de4b7be2ed34036868c692ab091b3c8`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:12.323678 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f799b936-c8cf-4321-b0db-3c59621772af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:12.354972 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:12.405902 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c8376-2ed4-4151-b3ef-d4fbb994bf4e admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:12.410313 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6d9a4-6eb7-42c0-b599-7ff3614060fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:12.410983 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6d9a4-6eb7-42c0-b599-7ff3614060fc service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:12.411452 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6d9a4-6eb7-42c0-b599-7ff3614060fc service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:12.411581 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6d9a4-6eb7-42c0-b599-7ff3614060fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:12.422707 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6d9a4-6eb7-42c0-b599-7ff3614060fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:12.424840 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6d9a4-6eb7-42c0-b599-7ff3614060fc service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:12.439353 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1062/2101] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:12 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:12.440386 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6d9a4-6eb7-42c0-b599-7ff3614060fc service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:12.442148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-54f6d9a4-6eb7-42c0-b599-7ff3614060fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:12.455644 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1040/2102] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:12.456601 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799b936-c8cf-4321-b0db-3c59621772af None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:12.463059 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6d9a4-6eb7-42c0-b599-7ff3614060fc service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:12.468294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c808e6-a3e6-4004-91f8-9ab569e2be0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:12.470341 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c808e6-a3e6-4004-91f8-9ab569e2be0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:12.470788 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c808e6-a3e6-4004-91f8-9ab569e2be0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:12.471100 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c808e6-a3e6-4004-91f8-9ab569e2be0f admin admin] PATH_INFO: `/v3/projects/f77dacc5ddd542128bc051e043b98632` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:12.472581 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c808e6-a3e6-4004-91f8-9ab569e2be0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f77dacc5ddd542128bc051e043b98632)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:12.474853 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c808e6-a3e6-4004-91f8-9ab569e2be0f admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:12.483282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799b936-c8cf-4321-b0db-3c59621772af None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:12.497030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348278a4-824f-438f-bd7d-c11a5887eb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:12.497860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348278a4-824f-438f-bd7d-c11a5887eb01 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:12.498152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348278a4-824f-438f-bd7d-c11a5887eb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:12.498435 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348278a4-824f-438f-bd7d-c11a5887eb01 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:12.499426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348278a4-824f-438f-bd7d-c11a5887eb01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:12.501251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348278a4-824f-438f-bd7d-c11a5887eb01 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:12.502901 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:12.519025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1041/2103] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:12.519025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348278a4-824f-438f-bd7d-c11a5887eb01 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:12.519517 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-348278a4-824f-438f-bd7d-c11a5887eb01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:12.541422 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348278a4-824f-438f-bd7d-c11a5887eb01 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:12.560274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-d4c808e6-a3e6-4004-91f8-9ab569e2be0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f77dacc5ddd542128bc051e043b98632', 'request_id': 'req-d4c808e6-a3e6-4004-91f8-9ab569e2be0f'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:12.573081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-d4c808e6-a3e6-4004-91f8-9ab569e2be0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f77dacc5ddd542128bc051e043b98632', 'request_id': 'req-d4c808e6-a3e6-4004-91f8-9ab569e2be0f'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:12.589673 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-d4c808e6-a3e6-4004-91f8-9ab569e2be0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f77dacc5ddd542128bc051e043b98632', 'request_id': 'req-d4c808e6-a3e6-4004-91f8-9ab569e2be0f'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:12.605498 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-d4c808e6-a3e6-4004-91f8-9ab569e2be0f admin admin] The token cache is being invalidate because project f77dacc5ddd542128bc051e043b98632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:12.605885 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-d4c808e6-a3e6-4004-91f8-9ab569e2be0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:12.606576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-d4c808e6-a3e6-4004-91f8-9ab569e2be0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:12.608516 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1063/2104] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:12 2022] DELETE /identity/v3/projects/f77dacc5ddd542128bc051e043b98632 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:12.609402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c808e6-a3e6-4004-91f8-9ab569e2be0f admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:12.611886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-d4c808e6-a3e6-4004-91f8-9ab569e2be0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:12.657372 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f4234-3afe-47b4-9508-f7be15877e05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:12.658532 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f4234-3afe-47b4-9508-f7be15877e05 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:12.658818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f4234-3afe-47b4-9508-f7be15877e05 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:12.659096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f4234-3afe-47b4-9508-f7be15877e05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:12.661326 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-527f4234-3afe-47b4-9508-f7be15877e05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:12.702162 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c808e6-a3e6-4004-91f8-9ab569e2be0f admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:12.707216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412dad80-6c87-4aa5-ae18-4b23d533e90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:12.708818 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412dad80-6c87-4aa5-ae18-4b23d533e90c admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:12.709113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412dad80-6c87-4aa5-ae18-4b23d533e90c admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:12.709465 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412dad80-6c87-4aa5-ae18-4b23d533e90c admin admin] PATH_INFO: `/v3/projects/7db76c8d852e48e38d013a29f190a9ba/users/c855226ea24b40d085d082d197f8f176/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:12.731028 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412dad80-6c87-4aa5-ae18-4b23d533e90c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7db76c8d852e48e38d013a29f190a9ba, user_id=c855226ea24b40d085d082d197f8f176, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:12.733860 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412dad80-6c87-4aa5-ae18-4b23d533e90c admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:12.742370 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f4234-3afe-47b4-9508-f7be15877e05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:12.744045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f4234-3afe-47b4-9508-f7be15877e05 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:12.751335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1064/2105] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:12 2022] PUT /identity/v3/projects/7db76c8d852e48e38d013a29f190a9ba/users/c855226ea24b40d085d082d197f8f176/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:12.752416 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412dad80-6c87-4aa5-ae18-4b23d533e90c admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:12.754579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-412dad80-6c87-4aa5-ae18-4b23d533e90c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:12.758865 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1042/2106] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:12.760579 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1bf84-d24d-4de0-8989-6bb7153a746b None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:12.760870 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1bf84-d24d-4de0-8989-6bb7153a746b None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:12.761145 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1bf84-d24d-4de0-8989-6bb7153a746b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:12.783954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-21d1bf84-d24d-4de0-8989-6bb7153a746b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:12.785189 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-21d1bf84-d24d-4de0-8989-6bb7153a746b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:12.871198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:12.883460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412dad80-6c87-4aa5-ae18-4b23d533e90c admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:12.889544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5562ed2-db48-4d18-b5b6-dba4056f87a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:12.890258 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5562ed2-db48-4d18-b5b6-dba4056f87a0 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:12.890547 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5562ed2-db48-4d18-b5b6-dba4056f87a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:12.890831 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5562ed2-db48-4d18-b5b6-dba4056f87a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:12.892499 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1043/2107] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:12.894627 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8a36a-982b-423d-8317-fac44c988d8b None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:12.895350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8a36a-982b-423d-8317-fac44c988d8b None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:12.897675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8a36a-982b-423d-8317-fac44c988d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:12.898914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-a5562ed2-db48-4d18-b5b6-dba4056f87a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:12.935478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-d8d8a36a-982b-423d-8317-fac44c988d8b None None] MFA Rules not processed for user `c855226ea24b40d085d082d197f8f176`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:12.936951 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d8d8a36a-982b-423d-8317-fac44c988d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:12.967038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:13.005722 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5562ed2-db48-4d18-b5b6-dba4056f87a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:13.007820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5562ed2-db48-4d18-b5b6-dba4056f87a0 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:13.036773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1065/2108] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:13.040776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1d9e6-bba4-4d28-8e4c-17d57b772700 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:13.041088 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1d9e6-bba4-4d28-8e4c-17d57b772700 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:13.042360 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1044/2109] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:13.042569 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1d9e6-bba4-4d28-8e4c-17d57b772700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:13.049828 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8a36a-982b-423d-8317-fac44c988d8b None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:13.085083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8a36a-982b-423d-8317-fac44c988d8b None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:13.095431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029613ce-d0b9-4ae8-a7cf-8b9920ec8989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:13.097135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-48d1d9e6-bba4-4d28-8e4c-17d57b772700 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:13.102529 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-48d1d9e6-bba4-4d28-8e4c-17d57b772700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:13.103020 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029613ce-d0b9-4ae8-a7cf-8b9920ec8989 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:13.103394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029613ce-d0b9-4ae8-a7cf-8b9920ec8989 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:13.103682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029613ce-d0b9-4ae8-a7cf-8b9920ec8989 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:13.104969 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029613ce-d0b9-4ae8-a7cf-8b9920ec8989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:13.108808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029613ce-d0b9-4ae8-a7cf-8b9920ec8989 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:13.130791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1066/2110] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:13.131812 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1d9e6-bba4-4d28-8e4c-17d57b772700 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:13.151057 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1d9e6-bba4-4d28-8e4c-17d57b772700 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:13.166691 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd87be-56b1-45c3-ab15-e9285c1954ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:13.167137 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd87be-56b1-45c3-ab15-e9285c1954ca admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:13.167432 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd87be-56b1-45c3-ab15-e9285c1954ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:13.167704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd87be-56b1-45c3-ab15-e9285c1954ca admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:13.174581 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd87be-56b1-45c3-ab15-e9285c1954ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:13.174933 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd87be-56b1-45c3-ab15-e9285c1954ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:13.176465 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1045/2111] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:13 2022] POST /identity/v3/projects => generated 369 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:13.176681 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd87be-56b1-45c3-ab15-e9285c1954ca admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:13.178155 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029613ce-d0b9-4ae8-a7cf-8b9920ec8989 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:13.213163 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1067/2112] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:49:13.214726 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd87be-56b1-45c3-ab15-e9285c1954ca admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:13.243354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd87be-56b1-45c3-ab15-e9285c1954ca admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:13.392392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d4270-d3d4-4343-87e8-f14bfef90a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:13.395269 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d4270-d3d4-4343-87e8-f14bfef90a5c admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:13.396069 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029613ce-d0b9-4ae8-a7cf-8b9920ec8989 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:13.396818 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d4270-d3d4-4343-87e8-f14bfef90a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:13.397200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d4270-d3d4-4343-87e8-f14bfef90a5c admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:13.399098 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d4270-d3d4-4343-87e8-f14bfef90a5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:13.402375 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d4270-d3d4-4343-87e8-f14bfef90a5c admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:13.410206 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459e0e9-e5d0-44cf-a1cc-4179a3dbba1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:13.411009 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459e0e9-e5d0-44cf-a1cc-4179a3dbba1f admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:13.411420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459e0e9-e5d0-44cf-a1cc-4179a3dbba1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:13.412229 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459e0e9-e5d0-44cf-a1cc-4179a3dbba1f admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:13.413017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459e0e9-e5d0-44cf-a1cc-4179a3dbba1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:13.423796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459e0e9-e5d0-44cf-a1cc-4179a3dbba1f admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:13.468612 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1068/2113] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:13 2022] POST /identity/v3/projects => generated 371 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:13.469983 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d4270-d3d4-4343-87e8-f14bfef90a5c admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:13.492995 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1046/2114] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:13 2022] POST /identity/v3/users => generated 291 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:13.494071 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459e0e9-e5d0-44cf-a1cc-4179a3dbba1f admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:13.526514 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459e0e9-e5d0-44cf-a1cc-4179a3dbba1f admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:13.631771 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d4270-d3d4-4343-87e8-f14bfef90a5c admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:13.653934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:13.654943 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101b3b4-dc5f-4acf-b372-f46829acb7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:13.655773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101b3b4-dc5f-4acf-b372-f46829acb7b8 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:13.656093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101b3b4-dc5f-4acf-b372-f46829acb7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:13.656467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101b3b4-dc5f-4acf-b372-f46829acb7b8 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:13.661774 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101b3b4-dc5f-4acf-b372-f46829acb7b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:13.668317 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101b3b4-dc5f-4acf-b372-f46829acb7b8 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:13.671934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19091de8-adff-4ce4-8668-4ac80468010f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:13.672582 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19091de8-adff-4ce4-8668-4ac80468010f admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:13.672884 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19091de8-adff-4ce4-8668-4ac80468010f admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:13.673144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19091de8-adff-4ce4-8668-4ac80468010f admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:13.673941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19091de8-adff-4ce4-8668-4ac80468010f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:13.675540 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19091de8-adff-4ce4-8668-4ac80468010f admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:13.715507 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1069/2115] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:13 2022] POST /identity/v3/users => generated 286 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:13.719858 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101b3b4-dc5f-4acf-b372-f46829acb7b8 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:13.724879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1047/2116] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:49:13.729670 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19091de8-adff-4ce4-8668-4ac80468010f admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:13.744255 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19091de8-adff-4ce4-8668-4ac80468010f admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:13.759609 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f34e8-4a34-4688-9b5d-21256c77e5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:13.760787 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f34e8-4a34-4688-9b5d-21256c77e5b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:13.761238 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f34e8-4a34-4688-9b5d-21256c77e5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:13.761587 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f34e8-4a34-4688-9b5d-21256c77e5b1 admin admin] PATH_INFO: `/v3/projects/8dacfca634af43a7acfba480a2c7678c/users/69d052de2ebe4ce698ad4c12be376048/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:13.769081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101b3b4-dc5f-4acf-b372-f46829acb7b8 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:13.785825 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603624c-fe8a-48f6-97f3-216f398e3496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:13.787263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603624c-fe8a-48f6-97f3-216f398e3496 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:13.788303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603624c-fe8a-48f6-97f3-216f398e3496 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:13.788644 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603624c-fe8a-48f6-97f3-216f398e3496 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:13.789881 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603624c-fe8a-48f6-97f3-216f398e3496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:13.791721 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603624c-fe8a-48f6-97f3-216f398e3496 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:13.799794 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f34e8-4a34-4688-9b5d-21256c77e5b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dacfca634af43a7acfba480a2c7678c, user_id=69d052de2ebe4ce698ad4c12be376048, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:13.805853 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f34e8-4a34-4688-9b5d-21256c77e5b1 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:13.814964 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1070/2117] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:49:13.819255 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603624c-fe8a-48f6-97f3-216f398e3496 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:13.830882 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1048/2118] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:13 2022] PUT /identity/v3/projects/8dacfca634af43a7acfba480a2c7678c/users/69d052de2ebe4ce698ad4c12be376048/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:49:13.834692 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603624c-fe8a-48f6-97f3-216f398e3496 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:13.835032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f34e8-4a34-4688-9b5d-21256c77e5b1 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:13.875973 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:13.920388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f34e8-4a34-4688-9b5d-21256c77e5b1 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:13.925441 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78a784-599c-4985-ae34-680bbe9b579b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:13.926170 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78a784-599c-4985-ae34-680bbe9b579b admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:13.926490 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78a784-599c-4985-ae34-680bbe9b579b admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:13.927221 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78a784-599c-4985-ae34-680bbe9b579b admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:13.927980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a78a784-599c-4985-ae34-680bbe9b579b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:13.929730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a78a784-599c-4985-ae34-680bbe9b579b admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:13.934819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844ad7c-8178-44cc-abc6-e0158239b819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:13.936679 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844ad7c-8178-44cc-abc6-e0158239b819 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:13.937039 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844ad7c-8178-44cc-abc6-e0158239b819 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:13.937455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844ad7c-8178-44cc-abc6-e0158239b819 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:13.938685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844ad7c-8178-44cc-abc6-e0158239b819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:13.940722 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844ad7c-8178-44cc-abc6-e0158239b819 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:13.942498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1049/2119] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:13.947542 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78a784-599c-4985-ae34-680bbe9b579b admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:13.964516 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78a784-599c-4985-ae34-680bbe9b579b admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:13.965470 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1071/2120] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:49:13.969630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844ad7c-8178-44cc-abc6-e0158239b819 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:13.970235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95483a1-4be5-4151-a5f6-deff56d3e3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:13.971291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95483a1-4be5-4151-a5f6-deff56d3e3e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:13.972385 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95483a1-4be5-4151-a5f6-deff56d3e3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:13.972385 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95483a1-4be5-4151-a5f6-deff56d3e3e9 admin admin] PATH_INFO: `/v3/projects/8dacfca634af43a7acfba480a2c7678c/users/69d052de2ebe4ce698ad4c12be376048/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:13.975539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95483a1-4be5-4151-a5f6-deff56d3e3e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dacfca634af43a7acfba480a2c7678c, user_id=69d052de2ebe4ce698ad4c12be376048, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:13.978631 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95483a1-4be5-4151-a5f6-deff56d3e3e9 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:13.986966 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844ad7c-8178-44cc-abc6-e0158239b819 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:13.993027 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1050/2121] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:13 2022] PUT /identity/v3/projects/8dacfca634af43a7acfba480a2c7678c/users/69d052de2ebe4ce698ad4c12be376048/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:49:14.005116 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5db5d-eafe-4585-9917-725a39080487 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:14.006078 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5db5d-eafe-4585-9917-725a39080487 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:14.006366 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5db5d-eafe-4585-9917-725a39080487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:14.057453 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-c8d5db5d-eafe-4585-9917-725a39080487 None None] MFA Rules not processed for user `69d052de2ebe4ce698ad4c12be376048`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:14.059046 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-c8d5db5d-eafe-4585-9917-725a39080487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:14.086507 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa8ce5-a8cf-4551-9499-12595b22155f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:14.087922 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa8ce5-a8cf-4551-9499-12595b22155f admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:14.088348 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa8ce5-a8cf-4551-9499-12595b22155f admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:14.088648 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa8ce5-a8cf-4551-9499-12595b22155f admin admin] PATH_INFO: `/v3/projects/aabb1607668d44cd9efda8922d62d5f4/users/68241c4c2bea4a10a5c5b7fcc3e0ff79/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:14.105254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa8ce5-a8cf-4551-9499-12595b22155f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aabb1607668d44cd9efda8922d62d5f4, user_id=68241c4c2bea4a10a5c5b7fcc3e0ff79, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:14.107930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aa8ce5-a8cf-4551-9499-12595b22155f admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:14.121559 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1072/2122] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:13 2022] PUT /identity/v3/projects/aabb1607668d44cd9efda8922d62d5f4/users/68241c4c2bea4a10a5c5b7fcc3e0ff79/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:14.133766 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc981e8-1283-4c5b-b284-5ca686a1cb5d None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:14.134062 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc981e8-1283-4c5b-b284-5ca686a1cb5d None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:14.134340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc981e8-1283-4c5b-b284-5ca686a1cb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:14.139675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1051/2123] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:14 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:14.152754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5db5d-eafe-4585-9917-725a39080487 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:14.154245 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-c8d5db5d-eafe-4585-9917-725a39080487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:14.163261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-1dc981e8-1283-4c5b-b284-5ca686a1cb5d None None] MFA Rules not processed for user `68241c4c2bea4a10a5c5b7fcc3e0ff79`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:14.164478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1dc981e8-1283-4c5b-b284-5ca686a1cb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:14.191517 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:14.217926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5db5d-eafe-4585-9917-725a39080487 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:14.230935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1073/2124] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:14.242088 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc981e8-1283-4c5b-b284-5ca686a1cb5d None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:14.273927 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529840c-f960-4526-a4b6-da072c699f36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:14.274629 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529840c-f960-4526-a4b6-da072c699f36 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:14.274948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529840c-f960-4526-a4b6-da072c699f36 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:14.275195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529840c-f960-4526-a4b6-da072c699f36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:14.289017 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:14.306874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc981e8-1283-4c5b-b284-5ca686a1cb5d None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:14.310818 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2befe-7346-473d-bcb8-39200a2bc0b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:14.311461 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2befe-7346-473d-bcb8-39200a2bc0b0 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:14.311743 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2befe-7346-473d-bcb8-39200a2bc0b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:14.312017 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2befe-7346-473d-bcb8-39200a2bc0b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:14.321485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2befe-7346-473d-bcb8-39200a2bc0b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:14.322964 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2befe-7346-473d-bcb8-39200a2bc0b0 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:14.332321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1074/2125] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:49:14.341303 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529840c-f960-4526-a4b6-da072c699f36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:14.342689 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529840c-f960-4526-a4b6-da072c699f36 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:14.353803 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1052/2126] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:14 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:49:14.362601 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2befe-7346-473d-bcb8-39200a2bc0b0 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:14.431835 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529840c-f960-4526-a4b6-da072c699f36 service cinder] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:14.442019 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529840c-f960-4526-a4b6-da072c699f36 service cinder] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:14.449102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16782ddf-3d4a-4eaa-9f33-163f0084862d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:14.449787 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16782ddf-3d4a-4eaa-9f33-163f0084862d service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:14.450075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16782ddf-3d4a-4eaa-9f33-163f0084862d service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:14.450352 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16782ddf-3d4a-4eaa-9f33-163f0084862d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:14.471154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2befe-7346-473d-bcb8-39200a2bc0b0 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:14.481546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7410d-6d82-41a3-b5d4-b09e5dbaff69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:14.481546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7410d-6d82-41a3-b5d4-b09e5dbaff69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:14.481546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7410d-6d82-41a3-b5d4-b09e5dbaff69 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:14.481546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7410d-6d82-41a3-b5d4-b09e5dbaff69 admin admin] PATH_INFO: `/v3/users/68241c4c2bea4a10a5c5b7fcc3e0ff79` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:14.481979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7410d-6d82-41a3-b5d4-b09e5dbaff69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68241c4c2bea4a10a5c5b7fcc3e0ff79)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:14.483618 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7410d-6d82-41a3-b5d4-b09e5dbaff69 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:14.558765 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16782ddf-3d4a-4eaa-9f33-163f0084862d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:14.560213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16782ddf-3d4a-4eaa-9f33-163f0084862d service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:14.563524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-92e7410d-6d82-41a3-b5d4-b09e5dbaff69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68241c4c2bea4a10a5c5b7fcc3e0ff79'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:14.574496 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1053/2127] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:14 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:14.581758 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-92e7410d-6d82-41a3-b5d4-b09e5dbaff69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68241c4c2bea4a10a5c5b7fcc3e0ff79'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:14.582858 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-92e7410d-6d82-41a3-b5d4-b09e5dbaff69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68241c4c2bea4a10a5c5b7fcc3e0ff79'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:14.594661 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-92e7410d-6d82-41a3-b5d4-b09e5dbaff69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68241c4c2bea4a10a5c5b7fcc3e0ff79'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:14.608003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-92e7410d-6d82-41a3-b5d4-b09e5dbaff69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68241c4c2bea4a10a5c5b7fcc3e0ff79'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:14.619148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1075/2128] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:14 2022] DELETE /identity/v3/users/68241c4c2bea4a10a5c5b7fcc3e0ff79 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:14.624018 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73acaf4-99ed-4ee5-945f-ce3d4e0ba193 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:14.624318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73acaf4-99ed-4ee5-945f-ce3d4e0ba193 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:14.624610 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73acaf4-99ed-4ee5-945f-ce3d4e0ba193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:14.643835 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-e73acaf4-99ed-4ee5-945f-ce3d4e0ba193 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:14.645056 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e73acaf4-99ed-4ee5-945f-ce3d4e0ba193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:14.681373 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:14.718092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1054/2129] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:14.729708 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7410d-6d82-41a3-b5d4-b09e5dbaff69 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:14.731146 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-92e7410d-6d82-41a3-b5d4-b09e5dbaff69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:14.804609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7410d-6d82-41a3-b5d4-b09e5dbaff69 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:14.809548 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f4ed5-2e68-4c02-be72-c95287aa8369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:14.812209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f4ed5-2e68-4c02-be72-c95287aa8369 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:14.812209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f4ed5-2e68-4c02-be72-c95287aa8369 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:14.812607 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f4ed5-2e68-4c02-be72-c95287aa8369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:14.842786 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f4ed5-2e68-4c02-be72-c95287aa8369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:14.844245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f4ed5-2e68-4c02-be72-c95287aa8369 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:14.850658 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:14.858776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1076/2130] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:15.194600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73acaf4-99ed-4ee5-945f-ce3d4e0ba193 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:15.198547 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e73acaf4-99ed-4ee5-945f-ce3d4e0ba193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:15.215598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73acaf4-99ed-4ee5-945f-ce3d4e0ba193 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:15.220581 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda2f96-77fc-463a-8c6f-575871664ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:15.221613 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda2f96-77fc-463a-8c6f-575871664ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:15.222020 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda2f96-77fc-463a-8c6f-575871664ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:15.222374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda2f96-77fc-463a-8c6f-575871664ad3 admin admin] PATH_INFO: `/v3/projects/aabb1607668d44cd9efda8922d62d5f4` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:15.223774 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda2f96-77fc-463a-8c6f-575871664ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aabb1607668d44cd9efda8922d62d5f4)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:15.226084 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda2f96-77fc-463a-8c6f-575871664ad3 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:15.276898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2bda2f96-77fc-463a-8c6f-575871664ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aabb1607668d44cd9efda8922d62d5f4', 'request_id': 'req-2bda2f96-77fc-463a-8c6f-575871664ad3'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:15.283693 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2bda2f96-77fc-463a-8c6f-575871664ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aabb1607668d44cd9efda8922d62d5f4', 'request_id': 'req-2bda2f96-77fc-463a-8c6f-575871664ad3'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:15.290323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2bda2f96-77fc-463a-8c6f-575871664ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aabb1607668d44cd9efda8922d62d5f4', 'request_id': 'req-2bda2f96-77fc-463a-8c6f-575871664ad3'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:15.305839 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2bda2f96-77fc-463a-8c6f-575871664ad3 admin admin] The token cache is being invalidate because project aabb1607668d44cd9efda8922d62d5f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:15.306174 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2bda2f96-77fc-463a-8c6f-575871664ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:15.307528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2bda2f96-77fc-463a-8c6f-575871664ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:15.309684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1055/2131] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:15 2022] DELETE /identity/v3/projects/aabb1607668d44cd9efda8922d62d5f4 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:15.314554 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc91ad3-4bce-4eef-b7f8-b09ea8688376 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:15.314900 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc91ad3-4bce-4eef-b7f8-b09ea8688376 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:15.315159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc91ad3-4bce-4eef-b7f8-b09ea8688376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:15.335370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-2bc91ad3-4bce-4eef-b7f8-b09ea8688376 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:15.336570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2bc91ad3-4bce-4eef-b7f8-b09ea8688376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:15.416801 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1077/2132] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:15.420045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d5562-5125-4954-bb32-d1decaf9164f None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:15.420511 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d5562-5125-4954-bb32-d1decaf9164f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:15.421022 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d5562-5125-4954-bb32-d1decaf9164f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:15.457348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-8f5d5562-5125-4954-bb32-d1decaf9164f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:15.460981 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-8f5d5562-5125-4954-bb32-d1decaf9164f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:15.474534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1056/2133] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:15.479163 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc91ad3-4bce-4eef-b7f8-b09ea8688376 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:15.490757 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc91ad3-4bce-4eef-b7f8-b09ea8688376 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:15.496075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93487dfb-ab60-4047-abc3-26dc1219bb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:15.496755 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93487dfb-ab60-4047-abc3-26dc1219bb97 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:15.497040 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93487dfb-ab60-4047-abc3-26dc1219bb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:15.497318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93487dfb-ab60-4047-abc3-26dc1219bb97 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:15.498093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93487dfb-ab60-4047-abc3-26dc1219bb97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:15.498404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93487dfb-ab60-4047-abc3-26dc1219bb97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:15.500046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93487dfb-ab60-4047-abc3-26dc1219bb97 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:15.513255 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1078/2134] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:49:15.517766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d5562-5125-4954-bb32-d1decaf9164f None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:15.528286 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d5562-5125-4954-bb32-d1decaf9164f None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:15.533224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47683728-19f6-4f3b-a9fd-209ca6dccb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:15.533926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47683728-19f6-4f3b-a9fd-209ca6dccb74 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:15.534214 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47683728-19f6-4f3b-a9fd-209ca6dccb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:15.534493 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47683728-19f6-4f3b-a9fd-209ca6dccb74 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:15.535447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47683728-19f6-4f3b-a9fd-209ca6dccb74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:15.538640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47683728-19f6-4f3b-a9fd-209ca6dccb74 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:15.567248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1057/2135] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:15 2022] POST /identity/v3/projects => generated 357 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:15.571690 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93487dfb-ab60-4047-abc3-26dc1219bb97 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:15.582460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93487dfb-ab60-4047-abc3-26dc1219bb97 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:15.647512 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e943b5f-fff1-4ede-b3e8-e617be814643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:15.648218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e943b5f-fff1-4ede-b3e8-e617be814643 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:15.648548 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e943b5f-fff1-4ede-b3e8-e617be814643 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:15.648816 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e943b5f-fff1-4ede-b3e8-e617be814643 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:15.649506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e943b5f-fff1-4ede-b3e8-e617be814643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:15.651473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e943b5f-fff1-4ede-b3e8-e617be814643 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:15.677104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1079/2136] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:15 2022] POST /identity/v3/users => generated 279 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:15.678330 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47683728-19f6-4f3b-a9fd-209ca6dccb74 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:15.690471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47683728-19f6-4f3b-a9fd-209ca6dccb74 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:15.696236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9e8b7-0c0e-4ce8-ab16-5efc957bf564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:15.696854 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9e8b7-0c0e-4ce8-ab16-5efc957bf564 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:15.697141 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9e8b7-0c0e-4ce8-ab16-5efc957bf564 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:15.697421 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9e8b7-0c0e-4ce8-ab16-5efc957bf564 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:15.698201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d9e8b7-0c0e-4ce8-ab16-5efc957bf564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:15.699864 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d9e8b7-0c0e-4ce8-ab16-5efc957bf564 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:15.713542 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1058/2137] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:15.718226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e943b5f-fff1-4ede-b3e8-e617be814643 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:15.729725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e943b5f-fff1-4ede-b3e8-e617be814643 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:15.735257 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac42764-26cc-4f72-a945-9ca1834e252a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:15.735945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac42764-26cc-4f72-a945-9ca1834e252a admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:15.736233 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac42764-26cc-4f72-a945-9ca1834e252a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:15.736501 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac42764-26cc-4f72-a945-9ca1834e252a admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:15.737274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac42764-26cc-4f72-a945-9ca1834e252a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:15.738919 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac42764-26cc-4f72-a945-9ca1834e252a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:15.754041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1080/2138] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:15.756105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9e8b7-0c0e-4ce8-ab16-5efc957bf564 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:15.766589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9e8b7-0c0e-4ce8-ab16-5efc957bf564 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:15.772720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffabfd-2264-43f7-8d6f-3d3cbd41b870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:15.773826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffabfd-2264-43f7-8d6f-3d3cbd41b870 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:15.774117 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffabfd-2264-43f7-8d6f-3d3cbd41b870 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:15.774398 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffabfd-2264-43f7-8d6f-3d3cbd41b870 admin admin] PATH_INFO: `/v3/projects/303d54c6379e4eaf885ceebfd2112286/users/58295c0c90d8447f8799e5ae7e0c7585/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:15.791028 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ffabfd-2264-43f7-8d6f-3d3cbd41b870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=303d54c6379e4eaf885ceebfd2112286, user_id=58295c0c90d8447f8799e5ae7e0c7585, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:15.793766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ffabfd-2264-43f7-8d6f-3d3cbd41b870 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:15.812675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1059/2139] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:15 2022] PUT /identity/v3/projects/303d54c6379e4eaf885ceebfd2112286/users/58295c0c90d8447f8799e5ae7e0c7585/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:15.823915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffa46e-94af-4850-a4b1-41018dcba2d5 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:15.824260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffa46e-94af-4850-a4b1-41018dcba2d5 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:15.824494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffa46e-94af-4850-a4b1-41018dcba2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:15.854111 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-4fffa46e-94af-4850-a4b1-41018dcba2d5 None None] MFA Rules not processed for user `58295c0c90d8447f8799e5ae7e0c7585`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:15.855336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-4fffa46e-94af-4850-a4b1-41018dcba2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:15.920191 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:15.932018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1081/2140] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:15 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:15.973033 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffabfd-2264-43f7-8d6f-3d3cbd41b870 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:15.976549 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-b9ffabfd-2264-43f7-8d6f-3d3cbd41b870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:16.055446 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffabfd-2264-43f7-8d6f-3d3cbd41b870 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:16.062028 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc595199-7178-45fa-b067-9b47db8737a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:16.062642 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc595199-7178-45fa-b067-9b47db8737a0 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:16.064036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc595199-7178-45fa-b067-9b47db8737a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:16.064450 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc595199-7178-45fa-b067-9b47db8737a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:16.080143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc595199-7178-45fa-b067-9b47db8737a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:16.082362 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc595199-7178-45fa-b067-9b47db8737a0 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:16.088349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:16.106818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1060/2141] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:15 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:16.117782 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffa46e-94af-4850-a4b1-41018dcba2d5 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:16.125155 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc595199-7178-45fa-b067-9b47db8737a0 service cinder] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:16.128423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-dc595199-7178-45fa-b067-9b47db8737a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:16.237104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffa46e-94af-4850-a4b1-41018dcba2d5 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:16.244528 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fdac54-7942-4caa-8d47-6b5936c55049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:16.245449 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdac54-7942-4caa-8d47-6b5936c55049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:16.245819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdac54-7942-4caa-8d47-6b5936c55049 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:16.246200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdac54-7942-4caa-8d47-6b5936c55049 admin admin] PATH_INFO: `/v3/users/58295c0c90d8447f8799e5ae7e0c7585` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:16.249454 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fdac54-7942-4caa-8d47-6b5936c55049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58295c0c90d8447f8799e5ae7e0c7585)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:16.251473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fdac54-7942-4caa-8d47-6b5936c55049 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:16.260539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc595199-7178-45fa-b067-9b47db8737a0 service cinder] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:16.267593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdd803-666f-45a3-97a6-01e064b21cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:16.269301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdd803-666f-45a3-97a6-01e064b21cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:16.269586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdd803-666f-45a3-97a6-01e064b21cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:16.270007 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdd803-666f-45a3-97a6-01e064b21cc5 admin admin] PATH_INFO: `/v3/users/188ff69c7ea54bcba44acdc1218363f8` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:16.271598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acdd803-666f-45a3-97a6-01e064b21cc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=188ff69c7ea54bcba44acdc1218363f8)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:16.273916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acdd803-666f-45a3-97a6-01e064b21cc5 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:16.335619 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-c6fdac54-7942-4caa-8d47-6b5936c55049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58295c0c90d8447f8799e5ae7e0c7585'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:16.367735 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-c6fdac54-7942-4caa-8d47-6b5936c55049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58295c0c90d8447f8799e5ae7e0c7585'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:16.370103 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-c6fdac54-7942-4caa-8d47-6b5936c55049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58295c0c90d8447f8799e5ae7e0c7585'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:16.384462 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-c6fdac54-7942-4caa-8d47-6b5936c55049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58295c0c90d8447f8799e5ae7e0c7585'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:16.395962 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2acdd803-666f-45a3-97a6-01e064b21cc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '188ff69c7ea54bcba44acdc1218363f8'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:16.397929 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-c6fdac54-7942-4caa-8d47-6b5936c55049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58295c0c90d8447f8799e5ae7e0c7585'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:16.409071 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1082/2142] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:16 2022] DELETE /identity/v3/users/58295c0c90d8447f8799e5ae7e0c7585 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:16.415432 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280dc3f-b53c-4793-a059-880346794875 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:16.416303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280dc3f-b53c-4793-a059-880346794875 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:16.416796 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280dc3f-b53c-4793-a059-880346794875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:16.422209 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2acdd803-666f-45a3-97a6-01e064b21cc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '188ff69c7ea54bcba44acdc1218363f8'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:16.433280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2acdd803-666f-45a3-97a6-01e064b21cc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '188ff69c7ea54bcba44acdc1218363f8'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:16.433934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2acdd803-666f-45a3-97a6-01e064b21cc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '188ff69c7ea54bcba44acdc1218363f8'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:16.434740 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2acdd803-666f-45a3-97a6-01e064b21cc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '188ff69c7ea54bcba44acdc1218363f8'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:16.445740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-a280dc3f-b53c-4793-a059-880346794875 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:16.447488 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-a280dc3f-b53c-4793-a059-880346794875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:16.471215 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1061/2143] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:16 2022] DELETE /identity/v3/users/188ff69c7ea54bcba44acdc1218363f8 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:49:16.476490 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38313882-8a87-453b-a92d-899384f35ad0 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:16.479094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38313882-8a87-453b-a92d-899384f35ad0 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:16.479094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38313882-8a87-453b-a92d-899384f35ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:16.512965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-38313882-8a87-453b-a92d-899384f35ad0 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:16.522205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-38313882-8a87-453b-a92d-899384f35ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:16.572861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1083/2144] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:16.590660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280dc3f-b53c-4793-a059-880346794875 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:16.592688 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-a280dc3f-b53c-4793-a059-880346794875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:16.622799 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1062/2145] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:16.628925 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:16.634145 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38313882-8a87-453b-a92d-899384f35ad0 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:16.637746 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-38313882-8a87-453b-a92d-899384f35ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:16.680995 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:16.736333 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280dc3f-b53c-4793-a059-880346794875 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:16.737421 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38313882-8a87-453b-a92d-899384f35ad0 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:16.741300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81f502-1e29-4f1f-8619-be72ac6eadd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:16.742242 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81f502-1e29-4f1f-8619-be72ac6eadd7 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:16.742844 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81f502-1e29-4f1f-8619-be72ac6eadd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:16.742959 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f05ba4-4648-4fd5-91dd-f7d0b7c38575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:16.743414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81f502-1e29-4f1f-8619-be72ac6eadd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:16.744201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f05ba4-4648-4fd5-91dd-f7d0b7c38575 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:16.744590 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f05ba4-4648-4fd5-91dd-f7d0b7c38575 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:16.745226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f05ba4-4648-4fd5-91dd-f7d0b7c38575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:16.756668 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81f502-1e29-4f1f-8619-be72ac6eadd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:16.758611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81f502-1e29-4f1f-8619-be72ac6eadd7 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:16.759669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f05ba4-4648-4fd5-91dd-f7d0b7c38575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:16.761744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f05ba4-4648-4fd5-91dd-f7d0b7c38575 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:16.771706 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1084/2146] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:16.782467 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1063/2147] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:17.184341 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81f502-1e29-4f1f-8619-be72ac6eadd7 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:17.187302 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0b81f502-1e29-4f1f-8619-be72ac6eadd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:17.208291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f05ba4-4648-4fd5-91dd-f7d0b7c38575 service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:17.210099 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f1f05ba4-4648-4fd5-91dd-f7d0b7c38575 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:17.219234 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81f502-1e29-4f1f-8619-be72ac6eadd7 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:17.225087 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2ee99-208e-4c1e-a497-a1cadf76222c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:17.225494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f05ba4-4648-4fd5-91dd-f7d0b7c38575 service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:17.225995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2ee99-208e-4c1e-a497-a1cadf76222c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:17.226264 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2ee99-208e-4c1e-a497-a1cadf76222c admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:17.226543 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2ee99-208e-4c1e-a497-a1cadf76222c admin admin] PATH_INFO: `/v3/projects/303d54c6379e4eaf885ceebfd2112286` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:17.228318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2ee99-208e-4c1e-a497-a1cadf76222c admin admin] RBAC: Authorizing `identity:delete_project(project_id=303d54c6379e4eaf885ceebfd2112286)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:17.229850 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2ee99-208e-4c1e-a497-a1cadf76222c admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:17.230610 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ed634-6a14-4419-8c3e-e9a849c04b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:17.231724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ed634-6a14-4419-8c3e-e9a849c04b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:17.232014 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ed634-6a14-4419-8c3e-e9a849c04b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:17.232297 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ed634-6a14-4419-8c3e-e9a849c04b5b admin admin] PATH_INFO: `/v3/projects/78b694283b9142afbedb2fd7b41293ec` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:17.233515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ed634-6a14-4419-8c3e-e9a849c04b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=78b694283b9142afbedb2fd7b41293ec)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:17.235761 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ed634-6a14-4419-8c3e-e9a849c04b5b admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:17.283850 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-d0c2ee99-208e-4c1e-a497-a1cadf76222c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '303d54c6379e4eaf885ceebfd2112286', 'request_id': 'req-d0c2ee99-208e-4c1e-a497-a1cadf76222c'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:17.287882 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-1f3ed634-6a14-4419-8c3e-e9a849c04b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78b694283b9142afbedb2fd7b41293ec', 'request_id': 'req-1f3ed634-6a14-4419-8c3e-e9a849c04b5b'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:17.290602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-d0c2ee99-208e-4c1e-a497-a1cadf76222c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '303d54c6379e4eaf885ceebfd2112286', 'request_id': 'req-d0c2ee99-208e-4c1e-a497-a1cadf76222c'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:17.294309 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-1f3ed634-6a14-4419-8c3e-e9a849c04b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78b694283b9142afbedb2fd7b41293ec', 'request_id': 'req-1f3ed634-6a14-4419-8c3e-e9a849c04b5b'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:17.298935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-d0c2ee99-208e-4c1e-a497-a1cadf76222c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '303d54c6379e4eaf885ceebfd2112286', 'request_id': 'req-d0c2ee99-208e-4c1e-a497-a1cadf76222c'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:17.301660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-1f3ed634-6a14-4419-8c3e-e9a849c04b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78b694283b9142afbedb2fd7b41293ec', 'request_id': 'req-1f3ed634-6a14-4419-8c3e-e9a849c04b5b'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:17.310591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-d0c2ee99-208e-4c1e-a497-a1cadf76222c admin admin] The token cache is being invalidate because project 303d54c6379e4eaf885ceebfd2112286 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:17.310879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-d0c2ee99-208e-4c1e-a497-a1cadf76222c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:17.311536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-d0c2ee99-208e-4c1e-a497-a1cadf76222c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:17.313094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1085/2148] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:17 2022] DELETE /identity/v3/projects/303d54c6379e4eaf885ceebfd2112286 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:17.315993 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-1f3ed634-6a14-4419-8c3e-e9a849c04b5b admin admin] The token cache is being invalidate because project 78b694283b9142afbedb2fd7b41293ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:17.316346 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-1f3ed634-6a14-4419-8c3e-e9a849c04b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:17.317015 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-1f3ed634-6a14-4419-8c3e-e9a849c04b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:17.319333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1064/2149] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:17 2022] DELETE /identity/v3/projects/78b694283b9142afbedb2fd7b41293ec => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:17.323485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2ee99-208e-4c1e-a497-a1cadf76222c admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:17.325045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-d0c2ee99-208e-4c1e-a497-a1cadf76222c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:17.398942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2ee99-208e-4c1e-a497-a1cadf76222c admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:17.449271 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:17.465955 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f06bca-9d5d-4a97-93b7-25f679639474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:17.466893 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f06bca-9d5d-4a97-93b7-25f679639474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:17.467182 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f06bca-9d5d-4a97-93b7-25f679639474 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:17.467466 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f06bca-9d5d-4a97-93b7-25f679639474 admin admin] PATH_INFO: `/v3/users/9de4b7be2ed34036868c692ab091b3c8` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:17.468811 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f06bca-9d5d-4a97-93b7-25f679639474 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9de4b7be2ed34036868c692ab091b3c8)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:17.470781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f06bca-9d5d-4a97-93b7-25f679639474 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:17.521453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-45f06bca-9d5d-4a97-93b7-25f679639474 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9de4b7be2ed34036868c692ab091b3c8'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:17.533326 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-45f06bca-9d5d-4a97-93b7-25f679639474 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9de4b7be2ed34036868c692ab091b3c8'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:17.534024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-45f06bca-9d5d-4a97-93b7-25f679639474 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9de4b7be2ed34036868c692ab091b3c8'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:17.541351 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-45f06bca-9d5d-4a97-93b7-25f679639474 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9de4b7be2ed34036868c692ab091b3c8'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:17.548364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-45f06bca-9d5d-4a97-93b7-25f679639474 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9de4b7be2ed34036868c692ab091b3c8'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:17.554754 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1086/2150] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:17 2022] DELETE /identity/v3/users/9de4b7be2ed34036868c692ab091b3c8 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:49:17.870893 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ed634-6a14-4419-8c3e-e9a849c04b5b admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:17.875320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1f3ed634-6a14-4419-8c3e-e9a849c04b5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:17.904863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ed634-6a14-4419-8c3e-e9a849c04b5b admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:17.981827 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82da685-20da-44ee-b766-48d0e87eac1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:17.982900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82da685-20da-44ee-b766-48d0e87eac1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:17.983566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82da685-20da-44ee-b766-48d0e87eac1b admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:17.983934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82da685-20da-44ee-b766-48d0e87eac1b admin admin] PATH_INFO: `/v3/projects/442ec400ec684463ac80c68137d33b67` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:17.986688 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82da685-20da-44ee-b766-48d0e87eac1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=442ec400ec684463ac80c68137d33b67)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:17.990606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82da685-20da-44ee-b766-48d0e87eac1b admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:18.048443 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:18.053013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d82da685-20da-44ee-b766-48d0e87eac1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '442ec400ec684463ac80c68137d33b67', 'request_id': 'req-d82da685-20da-44ee-b766-48d0e87eac1b'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:18.071155 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d82da685-20da-44ee-b766-48d0e87eac1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '442ec400ec684463ac80c68137d33b67', 'request_id': 'req-d82da685-20da-44ee-b766-48d0e87eac1b'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:18.071155 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d82da685-20da-44ee-b766-48d0e87eac1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '442ec400ec684463ac80c68137d33b67', 'request_id': 'req-d82da685-20da-44ee-b766-48d0e87eac1b'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:18.081397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d82da685-20da-44ee-b766-48d0e87eac1b admin admin] The token cache is being invalidate because project 442ec400ec684463ac80c68137d33b67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:18.081716 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d82da685-20da-44ee-b766-48d0e87eac1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:18.082349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d82da685-20da-44ee-b766-48d0e87eac1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:18.084327 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1065/2151] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:17 2022] DELETE /identity/v3/projects/442ec400ec684463ac80c68137d33b67 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:18.089511 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae4368-0a74-485b-8a8b-6e03ef076a3c None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:18.090044 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae4368-0a74-485b-8a8b-6e03ef076a3c None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:18.090284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae4368-0a74-485b-8a8b-6e03ef076a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:18.111697 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-adae4368-0a74-485b-8a8b-6e03ef076a3c None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:18.113539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-adae4368-0a74-485b-8a8b-6e03ef076a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:18.190714 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1087/2152] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:18.199520 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20b2ac-4bf0-4801-9bcc-e934e4f119f0 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:18.199809 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20b2ac-4bf0-4801-9bcc-e934e4f119f0 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:18.200161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20b2ac-4bf0-4801-9bcc-e934e4f119f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:18.221346 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-da20b2ac-4bf0-4801-9bcc-e934e4f119f0 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:18.222595 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-da20b2ac-4bf0-4801-9bcc-e934e4f119f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:18.234224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1066/2153] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:18.238416 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae4368-0a74-485b-8a8b-6e03ef076a3c None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:18.248323 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae4368-0a74-485b-8a8b-6e03ef076a3c None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:18.253987 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2386ec4-786f-4113-85b8-7358208b3b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:18.255539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2386ec4-786f-4113-85b8-7358208b3b50 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:18.255539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2386ec4-786f-4113-85b8-7358208b3b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:18.255729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2386ec4-786f-4113-85b8-7358208b3b50 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:18.256464 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2386ec4-786f-4113-85b8-7358208b3b50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:18.256782 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2386ec4-786f-4113-85b8-7358208b3b50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:18.258422 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2386ec4-786f-4113-85b8-7358208b3b50 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:18.273823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1088/2154] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:49:18.276733 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20b2ac-4bf0-4801-9bcc-e934e4f119f0 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:18.292541 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20b2ac-4bf0-4801-9bcc-e934e4f119f0 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:18.297997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7424d3-de08-4ec1-bf10-f350e29694b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:18.298762 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7424d3-de08-4ec1-bf10-f350e29694b8 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:18.299118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7424d3-de08-4ec1-bf10-f350e29694b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:18.299467 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7424d3-de08-4ec1-bf10-f350e29694b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:18.300242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7424d3-de08-4ec1-bf10-f350e29694b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:18.302383 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7424d3-de08-4ec1-bf10-f350e29694b8 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:18.330606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1067/2155] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:18 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:18.335669 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2386ec4-786f-4113-85b8-7358208b3b50 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:18.349992 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2386ec4-786f-4113-85b8-7358208b3b50 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:18.412918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:18.422036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629197a9-c192-43f9-b66d-baedb9d7feb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:18.422632 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629197a9-c192-43f9-b66d-baedb9d7feb0 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:18.423231 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629197a9-c192-43f9-b66d-baedb9d7feb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:18.423998 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629197a9-c192-43f9-b66d-baedb9d7feb0 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:18.424899 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629197a9-c192-43f9-b66d-baedb9d7feb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:18.435199 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629197a9-c192-43f9-b66d-baedb9d7feb0 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:18.457678 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1089/2156] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:18 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:49:18.457926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7424d3-de08-4ec1-bf10-f350e29694b8 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:18.460335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-8b7424d3-de08-4ec1-bf10-f350e29694b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:18.461548 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629197a9-c192-43f9-b66d-baedb9d7feb0 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:18.472347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629197a9-c192-43f9-b66d-baedb9d7feb0 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:18.473808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7424d3-de08-4ec1-bf10-f350e29694b8 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:18.477452 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b71ce1-1797-40c5-b2cf-f8496823b78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:18.478154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b71ce1-1797-40c5-b2cf-f8496823b78f admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:18.478443 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b71ce1-1797-40c5-b2cf-f8496823b78f admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:18.478680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c27d2-9e35-413b-b3aa-45d3efc0611f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:18.478791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b71ce1-1797-40c5-b2cf-f8496823b78f admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:18.479473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b71ce1-1797-40c5-b2cf-f8496823b78f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:18.479736 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c27d2-9e35-413b-b3aa-45d3efc0611f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:18.480090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c27d2-9e35-413b-b3aa-45d3efc0611f admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:18.480429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c27d2-9e35-413b-b3aa-45d3efc0611f admin admin] PATH_INFO: `/v3/users/c855226ea24b40d085d082d197f8f176` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:18.481136 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b71ce1-1797-40c5-b2cf-f8496823b78f admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:18.481805 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c27d2-9e35-413b-b3aa-45d3efc0611f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c855226ea24b40d085d082d197f8f176)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:18.483815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c27d2-9e35-413b-b3aa-45d3efc0611f admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:18.493495 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1090/2157] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:49:18.497457 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b71ce1-1797-40c5-b2cf-f8496823b78f admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:18.508942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b71ce1-1797-40c5-b2cf-f8496823b78f admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:18.514421 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72cc01-dd82-4cbe-ad8d-53070b1243f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:18.515168 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72cc01-dd82-4cbe-ad8d-53070b1243f6 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:18.515455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72cc01-dd82-4cbe-ad8d-53070b1243f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:18.515854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72cc01-dd82-4cbe-ad8d-53070b1243f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:18.516698 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72cc01-dd82-4cbe-ad8d-53070b1243f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:18.518356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72cc01-dd82-4cbe-ad8d-53070b1243f6 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:18.531817 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1091/2158] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:49:18.535288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-780c27d2-9e35-413b-b3aa-45d3efc0611f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c855226ea24b40d085d082d197f8f176'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:18.536083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72cc01-dd82-4cbe-ad8d-53070b1243f6 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:18.547127 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72cc01-dd82-4cbe-ad8d-53070b1243f6 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:18.548675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-780c27d2-9e35-413b-b3aa-45d3efc0611f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c855226ea24b40d085d082d197f8f176'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:18.552496 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd8380-4e99-4a7f-b65c-cda23c379fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:18.553425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd8380-4e99-4a7f-b65c-cda23c379fa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:18.553741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd8380-4e99-4a7f-b65c-cda23c379fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:18.554025 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd8380-4e99-4a7f-b65c-cda23c379fa8 admin admin] PATH_INFO: `/v3/projects/b1e68a0f9a7a4ebcb4c515afe5e37b11/users/d40e217bc6ff47c4b67bf9169bd61f14/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:18.559837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-780c27d2-9e35-413b-b3aa-45d3efc0611f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c855226ea24b40d085d082d197f8f176'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:18.560476 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-780c27d2-9e35-413b-b3aa-45d3efc0611f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c855226ea24b40d085d082d197f8f176'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:18.561103 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-780c27d2-9e35-413b-b3aa-45d3efc0611f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c855226ea24b40d085d082d197f8f176'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:18.571654 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd8380-4e99-4a7f-b65c-cda23c379fa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1e68a0f9a7a4ebcb4c515afe5e37b11, user_id=d40e217bc6ff47c4b67bf9169bd61f14, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:18.574337 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd8380-4e99-4a7f-b65c-cda23c379fa8 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:18.575018 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1068/2159] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:18 2022] DELETE /identity/v3/users/c855226ea24b40d085d082d197f8f176 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:18.579605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39c8fb-f9bc-4dbc-ba2f-165c244686de None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:18.579904 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39c8fb-f9bc-4dbc-ba2f-165c244686de None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:18.580257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39c8fb-f9bc-4dbc-ba2f-165c244686de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:18.588402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1092/2160] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:18 2022] PUT /identity/v3/projects/b1e68a0f9a7a4ebcb4c515afe5e37b11/users/d40e217bc6ff47c4b67bf9169bd61f14/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:49:18.599280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-4a39c8fb-f9bc-4dbc-ba2f-165c244686de None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:18.599970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9ffc6-ba4b-4a77-ab01-7ca79f4a0f77 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:18.600267 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9ffc6-ba4b-4a77-ab01-7ca79f4a0f77 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:18.600525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4a39c8fb-f9bc-4dbc-ba2f-165c244686de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:18.600633 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9ffc6-ba4b-4a77-ab01-7ca79f4a0f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:18.628775 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-24c9ffc6-ba4b-4a77-ab01-7ca79f4a0f77 None None] MFA Rules not processed for user `d40e217bc6ff47c4b67bf9169bd61f14`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:18.630888 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-24c9ffc6-ba4b-4a77-ab01-7ca79f4a0f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:18.664815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1069/2161] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:18.674807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39c8fb-f9bc-4dbc-ba2f-165c244686de None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:18.676439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4a39c8fb-f9bc-4dbc-ba2f-165c244686de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:18.701408 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:18.711575 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1093/2162] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:18.718516 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9ffc6-ba4b-4a77-ab01-7ca79f4a0f77 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:18.722175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-24c9ffc6-ba4b-4a77-ab01-7ca79f4a0f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:18.749803 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39c8fb-f9bc-4dbc-ba2f-165c244686de None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:18.754580 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54b95f-bdc3-44e7-84c7-03e7a931e66d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:18.755271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54b95f-bdc3-44e7-84c7-03e7a931e66d service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:18.755557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54b95f-bdc3-44e7-84c7-03e7a931e66d service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:18.755840 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54b95f-bdc3-44e7-84c7-03e7a931e66d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:18.765453 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54b95f-bdc3-44e7-84c7-03e7a931e66d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:18.766982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54b95f-bdc3-44e7-84c7-03e7a931e66d service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:18.777589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1070/2163] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:49:18.791020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9ffc6-ba4b-4a77-ab01-7ca79f4a0f77 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:18.794900 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5ca79-05dd-4c06-98ca-831c5b4960d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:18.795639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5ca79-05dd-4c06-98ca-831c5b4960d5 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:18.795999 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5ca79-05dd-4c06-98ca-831c5b4960d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:18.796601 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5ca79-05dd-4c06-98ca-831c5b4960d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:18.807944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5ca79-05dd-4c06-98ca-831c5b4960d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:18.809508 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5ca79-05dd-4c06-98ca-831c5b4960d5 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:18.821172 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1094/2164] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:49:19.463319 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54b95f-bdc3-44e7-84c7-03e7a931e66d service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:19.465051 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-eb54b95f-bdc3-44e7-84c7-03e7a931e66d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:19.480315 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54b95f-bdc3-44e7-84c7-03e7a931e66d service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:19.485530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22575ac0-4bc1-4e02-8027-e265bffa04c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:19.486524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22575ac0-4bc1-4e02-8027-e265bffa04c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:19.486808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22575ac0-4bc1-4e02-8027-e265bffa04c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:19.487080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22575ac0-4bc1-4e02-8027-e265bffa04c5 admin admin] PATH_INFO: `/v3/projects/7db76c8d852e48e38d013a29f190a9ba` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:19.488309 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22575ac0-4bc1-4e02-8027-e265bffa04c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7db76c8d852e48e38d013a29f190a9ba)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:19.497394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22575ac0-4bc1-4e02-8027-e265bffa04c5 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:19.573861 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-22575ac0-4bc1-4e02-8027-e265bffa04c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7db76c8d852e48e38d013a29f190a9ba', 'request_id': 'req-22575ac0-4bc1-4e02-8027-e265bffa04c5'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:19.585624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-22575ac0-4bc1-4e02-8027-e265bffa04c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7db76c8d852e48e38d013a29f190a9ba', 'request_id': 'req-22575ac0-4bc1-4e02-8027-e265bffa04c5'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:19.594425 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-22575ac0-4bc1-4e02-8027-e265bffa04c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7db76c8d852e48e38d013a29f190a9ba', 'request_id': 'req-22575ac0-4bc1-4e02-8027-e265bffa04c5'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:19.618253 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-22575ac0-4bc1-4e02-8027-e265bffa04c5 admin admin] The token cache is being invalidate because project 7db76c8d852e48e38d013a29f190a9ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:19.618831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-22575ac0-4bc1-4e02-8027-e265bffa04c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:19.619587 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-22575ac0-4bc1-4e02-8027-e265bffa04c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:19.621662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1071/2165] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:19 2022] DELETE /identity/v3/projects/7db76c8d852e48e38d013a29f190a9ba => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:49:19.627451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5ca79-05dd-4c06-98ca-831c5b4960d5 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:19.657006 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5ca79-05dd-4c06-98ca-831c5b4960d5 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:19.713078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:19.755780 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1354b-67c0-4308-8486-9f801acee22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:19.756921 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1354b-67c0-4308-8486-9f801acee22d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:19.757332 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1354b-67c0-4308-8486-9f801acee22d admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:19.757729 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1354b-67c0-4308-8486-9f801acee22d admin admin] PATH_INFO: `/v3/users/69d052de2ebe4ce698ad4c12be376048` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:19.759259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1354b-67c0-4308-8486-9f801acee22d admin admin] RBAC: Authorizing `identity:delete_user(user_id=69d052de2ebe4ce698ad4c12be376048)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:19.761359 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1354b-67c0-4308-8486-9f801acee22d admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:19.843657 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-90c1354b-67c0-4308-8486-9f801acee22d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69d052de2ebe4ce698ad4c12be376048'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:19.858181 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-90c1354b-67c0-4308-8486-9f801acee22d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69d052de2ebe4ce698ad4c12be376048'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:19.858978 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-90c1354b-67c0-4308-8486-9f801acee22d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69d052de2ebe4ce698ad4c12be376048'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:19.869244 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-90c1354b-67c0-4308-8486-9f801acee22d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69d052de2ebe4ce698ad4c12be376048'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:19.878545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-90c1354b-67c0-4308-8486-9f801acee22d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69d052de2ebe4ce698ad4c12be376048'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:19.889973 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1095/2166] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:19 2022] DELETE /identity/v3/users/69d052de2ebe4ce698ad4c12be376048 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:20.207242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22575ac0-4bc1-4e02-8027-e265bffa04c5 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:20.211237 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-22575ac0-4bc1-4e02-8027-e265bffa04c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:20.251351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22575ac0-4bc1-4e02-8027-e265bffa04c5 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:20.339987 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33686cbb-7412-46c4-8172-84ced3783f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:20.340873 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33686cbb-7412-46c4-8172-84ced3783f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:20.341155 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33686cbb-7412-46c4-8172-84ced3783f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:20.341430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33686cbb-7412-46c4-8172-84ced3783f3d admin admin] PATH_INFO: `/v3/projects/8dacfca634af43a7acfba480a2c7678c` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:20.342704 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33686cbb-7412-46c4-8172-84ced3783f3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dacfca634af43a7acfba480a2c7678c)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:20.345017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33686cbb-7412-46c4-8172-84ced3783f3d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:20.415085 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:20.418954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-33686cbb-7412-46c4-8172-84ced3783f3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dacfca634af43a7acfba480a2c7678c', 'request_id': 'req-33686cbb-7412-46c4-8172-84ced3783f3d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:20.429681 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-33686cbb-7412-46c4-8172-84ced3783f3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dacfca634af43a7acfba480a2c7678c', 'request_id': 'req-33686cbb-7412-46c4-8172-84ced3783f3d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:20.445422 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-33686cbb-7412-46c4-8172-84ced3783f3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dacfca634af43a7acfba480a2c7678c', 'request_id': 'req-33686cbb-7412-46c4-8172-84ced3783f3d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:20.480164 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-33686cbb-7412-46c4-8172-84ced3783f3d admin admin] The token cache is being invalidate because project 8dacfca634af43a7acfba480a2c7678c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:20.481726 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-33686cbb-7412-46c4-8172-84ced3783f3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:20.485198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-33686cbb-7412-46c4-8172-84ced3783f3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:20.499409 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1072/2167] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:20 2022] DELETE /identity/v3/projects/8dacfca634af43a7acfba480a2c7678c => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:20.501399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cb746-cb33-44bc-9d53-66d08dd7960a None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:20.501942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cb746-cb33-44bc-9d53-66d08dd7960a None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:20.502501 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cb746-cb33-44bc-9d53-66d08dd7960a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:20.557182 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-622cb746-cb33-44bc-9d53-66d08dd7960a None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:20.558558 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-622cb746-cb33-44bc-9d53-66d08dd7960a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:20.677943 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1096/2168] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:20.678679 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbb99f-ac84-4c77-bf3c-c22fbc70481d None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:20.678679 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbb99f-ac84-4c77-bf3c-c22fbc70481d None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:20.678878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbb99f-ac84-4c77-bf3c-c22fbc70481d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:20.712726 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-6ccbb99f-ac84-4c77-bf3c-c22fbc70481d None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:20.714264 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-6ccbb99f-ac84-4c77-bf3c-c22fbc70481d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:20.723507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1073/2169] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:20.727900 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cb746-cb33-44bc-9d53-66d08dd7960a None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:20.742120 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cb746-cb33-44bc-9d53-66d08dd7960a None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:20.748943 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163743f7-c343-40f6-ac68-fcad36644301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:20.749664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163743f7-c343-40f6-ac68-fcad36644301 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:20.749965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163743f7-c343-40f6-ac68-fcad36644301 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:20.753809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163743f7-c343-40f6-ac68-fcad36644301 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:20.753809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163743f7-c343-40f6-ac68-fcad36644301 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:20.753809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163743f7-c343-40f6-ac68-fcad36644301 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:20.756477 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163743f7-c343-40f6-ac68-fcad36644301 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:20.769552 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1097/2170] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:49:20.774556 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbb99f-ac84-4c77-bf3c-c22fbc70481d None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:20.786255 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbb99f-ac84-4c77-bf3c-c22fbc70481d None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:20.791640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b844c-fccc-49b6-976a-70875a0981e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:20.792307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b844c-fccc-49b6-976a-70875a0981e4 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:20.792588 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b844c-fccc-49b6-976a-70875a0981e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:20.792865 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b844c-fccc-49b6-976a-70875a0981e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:20.793573 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b844c-fccc-49b6-976a-70875a0981e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:20.795835 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b844c-fccc-49b6-976a-70875a0981e4 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:20.840493 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1074/2171] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:20 2022] POST /identity/v3/projects => generated 387 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:20.844571 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163743f7-c343-40f6-ac68-fcad36644301 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:20.857170 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163743f7-c343-40f6-ac68-fcad36644301 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:20.908749 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:20.940382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f22432-f417-48c5-aeb9-8a965c482fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:20.941151 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f22432-f417-48c5-aeb9-8a965c482fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:20.941345 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f22432-f417-48c5-aeb9-8a965c482fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:20.941629 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f22432-f417-48c5-aeb9-8a965c482fc8 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:20.942419 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f22432-f417-48c5-aeb9-8a965c482fc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:20.944361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f22432-f417-48c5-aeb9-8a965c482fc8 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:20.967243 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1098/2172] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:20 2022] POST /identity/v3/users => generated 294 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:49:20.971886 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b844c-fccc-49b6-976a-70875a0981e4 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:20.983656 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b844c-fccc-49b6-976a-70875a0981e4 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:20.988770 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3e03a-a703-4d75-bb9a-8835a417253d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:20.989487 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3e03a-a703-4d75-bb9a-8835a417253d admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:20.989944 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3e03a-a703-4d75-bb9a-8835a417253d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:20.990226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3e03a-a703-4d75-bb9a-8835a417253d admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:20.990962 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3e03a-a703-4d75-bb9a-8835a417253d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:20.992698 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3e03a-a703-4d75-bb9a-8835a417253d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:21.004463 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1075/2173] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:21.009198 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f22432-f417-48c5-aeb9-8a965c482fc8 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:21.019629 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f22432-f417-48c5-aeb9-8a965c482fc8 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:21.025152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bacc77-0e48-4042-ac55-b67977ae799d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:21.025854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bacc77-0e48-4042-ac55-b67977ae799d admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:21.026145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bacc77-0e48-4042-ac55-b67977ae799d admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:21.026428 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bacc77-0e48-4042-ac55-b67977ae799d admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:21.027210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bacc77-0e48-4042-ac55-b67977ae799d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:21.028907 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bacc77-0e48-4042-ac55-b67977ae799d admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:21.040297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1099/2174] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:21.044830 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3e03a-a703-4d75-bb9a-8835a417253d admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:21.059827 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3e03a-a703-4d75-bb9a-8835a417253d admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:21.067083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300361cb-b270-4489-bb14-a92abe389572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:21.068755 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300361cb-b270-4489-bb14-a92abe389572 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:21.069056 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300361cb-b270-4489-bb14-a92abe389572 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:21.069368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300361cb-b270-4489-bb14-a92abe389572 admin admin] PATH_INFO: `/v3/projects/9e77bfab874a4041922079deda6aee34/users/198fac65d6c544518eb47b6fe54acc82/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:21.086340 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300361cb-b270-4489-bb14-a92abe389572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e77bfab874a4041922079deda6aee34, user_id=198fac65d6c544518eb47b6fe54acc82, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:21.089121 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300361cb-b270-4489-bb14-a92abe389572 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:21.104862 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1076/2175] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:21 2022] PUT /identity/v3/projects/9e77bfab874a4041922079deda6aee34/users/198fac65d6c544518eb47b6fe54acc82/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:49:21.116098 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c456ec-2d00-4e30-afa6-a0c662cc00e1 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:21.116527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c456ec-2d00-4e30-afa6-a0c662cc00e1 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:21.116946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c456ec-2d00-4e30-afa6-a0c662cc00e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:21.152082 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-55c456ec-2d00-4e30-afa6-a0c662cc00e1 None None] MFA Rules not processed for user `198fac65d6c544518eb47b6fe54acc82`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:21.153355 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-55c456ec-2d00-4e30-afa6-a0c662cc00e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:21.225187 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1100/2176] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:21.229433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300361cb-b270-4489-bb14-a92abe389572 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:21.298681 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300361cb-b270-4489-bb14-a92abe389572 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:21.306451 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18d7fe-2b38-4137-a1f6-fd7f3ed7f5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:21.307159 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18d7fe-2b38-4137-a1f6-fd7f3ed7f5b8 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:21.307476 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18d7fe-2b38-4137-a1f6-fd7f3ed7f5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:21.307760 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18d7fe-2b38-4137-a1f6-fd7f3ed7f5b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:21.308464 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18d7fe-2b38-4137-a1f6-fd7f3ed7f5b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:21.310508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18d7fe-2b38-4137-a1f6-fd7f3ed7f5b8 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:21.347634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1077/2177] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:21 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:49:21.350725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c456ec-2d00-4e30-afa6-a0c662cc00e1 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:21.363674 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c456ec-2d00-4e30-afa6-a0c662cc00e1 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:21.445592 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49fcf6-5bb6-4608-8fcb-3971f4febced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:21.446431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49fcf6-5bb6-4608-8fcb-3971f4febced admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:21.446734 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49fcf6-5bb6-4608-8fcb-3971f4febced admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:21.447009 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49fcf6-5bb6-4608-8fcb-3971f4febced admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:21.447722 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49fcf6-5bb6-4608-8fcb-3971f4febced admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:21.449688 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49fcf6-5bb6-4608-8fcb-3971f4febced admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:21.481720 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1101/2178] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:21 2022] POST /identity/v3/users => generated 299 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:21.483617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18d7fe-2b38-4137-a1f6-fd7f3ed7f5b8 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:21.497982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18d7fe-2b38-4137-a1f6-fd7f3ed7f5b8 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:21.504556 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf3adc-7202-447c-90c5-15577e66406f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:21.505362 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf3adc-7202-447c-90c5-15577e66406f admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:21.505739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf3adc-7202-447c-90c5-15577e66406f admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:21.506160 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf3adc-7202-447c-90c5-15577e66406f admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:21.506908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf3adc-7202-447c-90c5-15577e66406f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:21.508731 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf3adc-7202-447c-90c5-15577e66406f admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:21.528405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1078/2179] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:21.532666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49fcf6-5bb6-4608-8fcb-3971f4febced admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:21.545208 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49fcf6-5bb6-4608-8fcb-3971f4febced admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:21.552123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2dcf6-732f-4ead-9400-e34ce3ae717b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:21.552923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2dcf6-732f-4ead-9400-e34ce3ae717b admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:21.553210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2dcf6-732f-4ead-9400-e34ce3ae717b admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:21.553912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2dcf6-732f-4ead-9400-e34ce3ae717b admin admin] PATH_INFO: `/v3/projects/a552ac26c5c14159a1c62889ead232c7/users/054d9c01cc80408084cbfe08418ef917/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:21.573132 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2dcf6-732f-4ead-9400-e34ce3ae717b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a552ac26c5c14159a1c62889ead232c7, user_id=054d9c01cc80408084cbfe08418ef917, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:21.576193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2dcf6-732f-4ead-9400-e34ce3ae717b admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:21.597529 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1102/2180] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:21 2022] PUT /identity/v3/projects/a552ac26c5c14159a1c62889ead232c7/users/054d9c01cc80408084cbfe08418ef917/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:21.599606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf3adc-7202-447c-90c5-15577e66406f admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:21.617888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf3adc-7202-447c-90c5-15577e66406f admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:21.655841 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:21.678160 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25699c58-8f0b-4f1c-aac0-600bf877bebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:21.678848 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25699c58-8f0b-4f1c-aac0-600bf877bebe admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:21.679143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25699c58-8f0b-4f1c-aac0-600bf877bebe admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:21.679424 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25699c58-8f0b-4f1c-aac0-600bf877bebe admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:21.680166 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25699c58-8f0b-4f1c-aac0-600bf877bebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:21.681987 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25699c58-8f0b-4f1c-aac0-600bf877bebe admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:21.699556 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1079/2181] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:21.699819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2dcf6-732f-4ead-9400-e34ce3ae717b admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:21.720476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2dcf6-732f-4ead-9400-e34ce3ae717b admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:21.726787 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c4602-4a6f-4e22-854f-0d575a3b2b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:21.727702 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c4602-4a6f-4e22-854f-0d575a3b2b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:21.727990 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c4602-4a6f-4e22-854f-0d575a3b2b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:21.728267 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c4602-4a6f-4e22-854f-0d575a3b2b41 admin admin] PATH_INFO: `/v3/projects/a552ac26c5c14159a1c62889ead232c7/users/054d9c01cc80408084cbfe08418ef917/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:21.730503 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c4602-4a6f-4e22-854f-0d575a3b2b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a552ac26c5c14159a1c62889ead232c7, user_id=054d9c01cc80408084cbfe08418ef917, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:21.734846 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c4602-4a6f-4e22-854f-0d575a3b2b41 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:21.740704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:21.747050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1103/2182] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:21 2022] PUT /identity/v3/projects/a552ac26c5c14159a1c62889ead232c7/users/054d9c01cc80408084cbfe08418ef917/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:21.760920 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084cff9-a83e-4917-8ee4-71a9188cd483 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:21.761516 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084cff9-a83e-4917-8ee4-71a9188cd483 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:21.764681 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084cff9-a83e-4917-8ee4-71a9188cd483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:21.808985 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-e084cff9-a83e-4917-8ee4-71a9188cd483 None None] MFA Rules not processed for user `054d9c01cc80408084cbfe08418ef917`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:21.810265 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e084cff9-a83e-4917-8ee4-71a9188cd483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:21.937699 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1080/2183] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:21.946175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c4602-4a6f-4e22-854f-0d575a3b2b41 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:21.948023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-768c4602-4a6f-4e22-854f-0d575a3b2b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:22.044660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c4602-4a6f-4e22-854f-0d575a3b2b41 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:22.055165 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e0420-c26d-43ed-b2f0-40151e54c663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:22.055844 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e0420-c26d-43ed-b2f0-40151e54c663 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:22.056157 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e0420-c26d-43ed-b2f0-40151e54c663 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:22.056666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e0420-c26d-43ed-b2f0-40151e54c663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:22.068450 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e0420-c26d-43ed-b2f0-40151e54c663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:22.070734 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e0420-c26d-43ed-b2f0-40151e54c663 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:22.083164 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1104/2184] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:22.153253 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084cff9-a83e-4917-8ee4-71a9188cd483 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:22.155101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e084cff9-a83e-4917-8ee4-71a9188cd483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:22.160586 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e0420-c26d-43ed-b2f0-40151e54c663 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:22.231733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e0420-c26d-43ed-b2f0-40151e54c663 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:22.236432 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084cff9-a83e-4917-8ee4-71a9188cd483 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:22.236659 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b9cb8-84af-4e55-98de-54efccedbbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:22.237538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b9cb8-84af-4e55-98de-54efccedbbd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:22.237906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b9cb8-84af-4e55-98de-54efccedbbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:22.238182 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b9cb8-84af-4e55-98de-54efccedbbd6 admin admin] PATH_INFO: `/v3/users/198fac65d6c544518eb47b6fe54acc82` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:22.239630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b9cb8-84af-4e55-98de-54efccedbbd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=198fac65d6c544518eb47b6fe54acc82)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:22.241125 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6ea94-2f50-41eb-915f-23df8aeaed58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:22.241482 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b9cb8-84af-4e55-98de-54efccedbbd6 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:22.242044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6ea94-2f50-41eb-915f-23df8aeaed58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:22.242332 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6ea94-2f50-41eb-915f-23df8aeaed58 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:22.242612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6ea94-2f50-41eb-915f-23df8aeaed58 admin admin] PATH_INFO: `/v3/users/d40e217bc6ff47c4b67bf9169bd61f14` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:22.243896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6ea94-2f50-41eb-915f-23df8aeaed58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d40e217bc6ff47c4b67bf9169bd61f14)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:22.245850 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6ea94-2f50-41eb-915f-23df8aeaed58 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:22.293739 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-360b9cb8-84af-4e55-98de-54efccedbbd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '198fac65d6c544518eb47b6fe54acc82'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:22.298735 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-02c6ea94-2f50-41eb-915f-23df8aeaed58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd40e217bc6ff47c4b67bf9169bd61f14'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:22.308036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-360b9cb8-84af-4e55-98de-54efccedbbd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '198fac65d6c544518eb47b6fe54acc82'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:22.308828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-360b9cb8-84af-4e55-98de-54efccedbbd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '198fac65d6c544518eb47b6fe54acc82'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:22.311738 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-02c6ea94-2f50-41eb-915f-23df8aeaed58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd40e217bc6ff47c4b67bf9169bd61f14'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:22.314693 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:22.318988 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-360b9cb8-84af-4e55-98de-54efccedbbd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '198fac65d6c544518eb47b6fe54acc82'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:22.322629 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-02c6ea94-2f50-41eb-915f-23df8aeaed58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd40e217bc6ff47c4b67bf9169bd61f14'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:22.323661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-02c6ea94-2f50-41eb-915f-23df8aeaed58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd40e217bc6ff47c4b67bf9169bd61f14'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:22.324182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-02c6ea94-2f50-41eb-915f-23df8aeaed58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd40e217bc6ff47c4b67bf9169bd61f14'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:22.327207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-360b9cb8-84af-4e55-98de-54efccedbbd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '198fac65d6c544518eb47b6fe54acc82'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:22.334556 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1105/2185] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:22 2022] DELETE /identity/v3/users/198fac65d6c544518eb47b6fe54acc82 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:22.340257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1081/2186] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:22 2022] DELETE /identity/v3/users/d40e217bc6ff47c4b67bf9169bd61f14 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:49:22.344081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca0fa5-9c20-4811-a298-a2ddf7879951 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:22.344380 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca0fa5-9c20-4811-a298-a2ddf7879951 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:22.344663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca0fa5-9c20-4811-a298-a2ddf7879951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:22.345488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b684a0-5a36-417b-9072-25e207d268af None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:22.345805 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b684a0-5a36-417b-9072-25e207d268af None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:22.346088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b684a0-5a36-417b-9072-25e207d268af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:22.363847 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-24ca0fa5-9c20-4811-a298-a2ddf7879951 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:22.366382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-24ca0fa5-9c20-4811-a298-a2ddf7879951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:22.368106 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-e1b684a0-5a36-417b-9072-25e207d268af None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:22.369277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e1b684a0-5a36-417b-9072-25e207d268af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:22.380672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:22.434496 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1082/2187] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:22.442089 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1106/2188] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:22.444882 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b684a0-5a36-417b-9072-25e207d268af None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:22.446528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e1b684a0-5a36-417b-9072-25e207d268af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:22.452713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca0fa5-9c20-4811-a298-a2ddf7879951 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:22.454216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-24ca0fa5-9c20-4811-a298-a2ddf7879951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:22.510896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b684a0-5a36-417b-9072-25e207d268af None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:22.514871 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d644a-34f1-4917-bfd2-c81ca9f535ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:22.515532 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d644a-34f1-4917-bfd2-c81ca9f535ad service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:22.515818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d644a-34f1-4917-bfd2-c81ca9f535ad service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:22.516098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d644a-34f1-4917-bfd2-c81ca9f535ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:22.518846 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca0fa5-9c20-4811-a298-a2ddf7879951 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:22.523325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def91fb6-eaf9-4d07-bd44-17d1d9c021e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:22.523980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def91fb6-eaf9-4d07-bd44-17d1d9c021e5 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:22.524265 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def91fb6-eaf9-4d07-bd44-17d1d9c021e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:22.524660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def91fb6-eaf9-4d07-bd44-17d1d9c021e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:22.526716 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d644a-34f1-4917-bfd2-c81ca9f535ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:22.528154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d644a-34f1-4917-bfd2-c81ca9f535ad service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:22.536798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def91fb6-eaf9-4d07-bd44-17d1d9c021e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:22.538274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def91fb6-eaf9-4d07-bd44-17d1d9c021e5 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:22.539418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1083/2189] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:49:22.548824 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1107/2190] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:23.312137 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d644a-34f1-4917-bfd2-c81ca9f535ad service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:23.313737 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-0f9d644a-34f1-4917-bfd2-c81ca9f535ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:23.325180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def91fb6-eaf9-4d07-bd44-17d1d9c021e5 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:23.326702 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-def91fb6-eaf9-4d07-bd44-17d1d9c021e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:23.329124 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d644a-34f1-4917-bfd2-c81ca9f535ad service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:23.335500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9f9d1-57cc-4b89-a709-996a1b936610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:23.336366 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9f9d1-57cc-4b89-a709-996a1b936610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:23.336644 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9f9d1-57cc-4b89-a709-996a1b936610 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:23.336925 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9f9d1-57cc-4b89-a709-996a1b936610 admin admin] PATH_INFO: `/v3/projects/9e77bfab874a4041922079deda6aee34` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:23.338209 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9f9d1-57cc-4b89-a709-996a1b936610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e77bfab874a4041922079deda6aee34)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:23.340817 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9f9d1-57cc-4b89-a709-996a1b936610 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:23.341244 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def91fb6-eaf9-4d07-bd44-17d1d9c021e5 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:23.346803 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dccbd-1065-479b-b777-4f8803648545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:23.347947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dccbd-1065-479b-b777-4f8803648545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:23.349056 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dccbd-1065-479b-b777-4f8803648545 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:23.349908 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dccbd-1065-479b-b777-4f8803648545 admin admin] PATH_INFO: `/v3/projects/b1e68a0f9a7a4ebcb4c515afe5e37b11` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:23.351542 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dccbd-1065-479b-b777-4f8803648545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1e68a0f9a7a4ebcb4c515afe5e37b11)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:23.353898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dccbd-1065-479b-b777-4f8803648545 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:23.437615 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-69f9f9d1-57cc-4b89-a709-996a1b936610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e77bfab874a4041922079deda6aee34', 'request_id': 'req-69f9f9d1-57cc-4b89-a709-996a1b936610'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:23.448104 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-69f9f9d1-57cc-4b89-a709-996a1b936610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e77bfab874a4041922079deda6aee34', 'request_id': 'req-69f9f9d1-57cc-4b89-a709-996a1b936610'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:23.456566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-69f9f9d1-57cc-4b89-a709-996a1b936610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e77bfab874a4041922079deda6aee34', 'request_id': 'req-69f9f9d1-57cc-4b89-a709-996a1b936610'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:23.463491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-227dccbd-1065-479b-b777-4f8803648545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1e68a0f9a7a4ebcb4c515afe5e37b11', 'request_id': 'req-227dccbd-1065-479b-b777-4f8803648545'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:23.475254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-227dccbd-1065-479b-b777-4f8803648545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1e68a0f9a7a4ebcb4c515afe5e37b11', 'request_id': 'req-227dccbd-1065-479b-b777-4f8803648545'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:23.484494 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-69f9f9d1-57cc-4b89-a709-996a1b936610 admin admin] The token cache is being invalidate because project 9e77bfab874a4041922079deda6aee34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:23.484661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-69f9f9d1-57cc-4b89-a709-996a1b936610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:23.486591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-69f9f9d1-57cc-4b89-a709-996a1b936610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:23.488100 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1084/2191] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:23 2022] DELETE /identity/v3/projects/9e77bfab874a4041922079deda6aee34 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:23.492628 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-227dccbd-1065-479b-b777-4f8803648545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1e68a0f9a7a4ebcb4c515afe5e37b11', 'request_id': 'req-227dccbd-1065-479b-b777-4f8803648545'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:23.492768 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9f9d1-57cc-4b89-a709-996a1b936610 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:23.494264 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-69f9f9d1-57cc-4b89-a709-996a1b936610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:23.505258 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-227dccbd-1065-479b-b777-4f8803648545 admin admin] The token cache is being invalidate because project b1e68a0f9a7a4ebcb4c515afe5e37b11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:23.506898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-227dccbd-1065-479b-b777-4f8803648545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:23.507607 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-227dccbd-1065-479b-b777-4f8803648545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:23.509601 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1108/2192] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:23 2022] DELETE /identity/v3/projects/b1e68a0f9a7a4ebcb4c515afe5e37b11 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:23.514285 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b51089-2bb5-4243-9be2-eb04085d064d None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:23.514657 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b51089-2bb5-4243-9be2-eb04085d064d None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:23.515097 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b51089-2bb5-4243-9be2-eb04085d064d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:23.515701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:23.551863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-37b51089-2bb5-4243-9be2-eb04085d064d None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:23.553146 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-37b51089-2bb5-4243-9be2-eb04085d064d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:23.575823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9f9d1-57cc-4b89-a709-996a1b936610 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:23.606594 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:23.636899 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1109/2193] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:23.639407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b23faf-1ec5-4306-ae9e-1fb6c51b461b None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:23.639963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b23faf-1ec5-4306-ae9e-1fb6c51b461b None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:23.640146 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b23faf-1ec5-4306-ae9e-1fb6c51b461b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:23.642894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59f3c0-8214-4492-9d31-6615cde245d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:23.644210 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59f3c0-8214-4492-9d31-6615cde245d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:23.644573 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59f3c0-8214-4492-9d31-6615cde245d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:23.644973 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59f3c0-8214-4492-9d31-6615cde245d5 admin admin] PATH_INFO: `/v3/users/054d9c01cc80408084cbfe08418ef917` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:23.647129 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59f3c0-8214-4492-9d31-6615cde245d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=054d9c01cc80408084cbfe08418ef917)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:23.649307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59f3c0-8214-4492-9d31-6615cde245d5 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:23.665788 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-42b23faf-1ec5-4306-ae9e-1fb6c51b461b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:23.668078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-42b23faf-1ec5-4306-ae9e-1fb6c51b461b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:23.685190 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1110/2194] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:23.689987 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b23faf-1ec5-4306-ae9e-1fb6c51b461b None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:23.700690 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b23faf-1ec5-4306-ae9e-1fb6c51b461b None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:23.706600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04885c9f-6de1-4a21-9b68-56885724bdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:23.707361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04885c9f-6de1-4a21-9b68-56885724bdca admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:23.708359 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04885c9f-6de1-4a21-9b68-56885724bdca admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:23.708722 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04885c9f-6de1-4a21-9b68-56885724bdca admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:23.709777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04885c9f-6de1-4a21-9b68-56885724bdca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:23.710164 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04885c9f-6de1-4a21-9b68-56885724bdca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:23.711874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04885c9f-6de1-4a21-9b68-56885724bdca admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:23.725476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1111/2195] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:49:23.729589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3f59f3c0-8214-4492-9d31-6615cde245d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '054d9c01cc80408084cbfe08418ef917'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:23.730213 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04885c9f-6de1-4a21-9b68-56885724bdca admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:23.741918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04885c9f-6de1-4a21-9b68-56885724bdca admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:23.747237 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc2156-03f4-4f1d-b077-c146b7fd0b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:23.747749 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc2156-03f4-4f1d-b077-c146b7fd0b1a admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:23.748032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc2156-03f4-4f1d-b077-c146b7fd0b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:23.748334 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc2156-03f4-4f1d-b077-c146b7fd0b1a admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:23.749023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc2156-03f4-4f1d-b077-c146b7fd0b1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:23.750665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3f59f3c0-8214-4492-9d31-6615cde245d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '054d9c01cc80408084cbfe08418ef917'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:23.751263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc2156-03f4-4f1d-b077-c146b7fd0b1a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:23.761391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3f59f3c0-8214-4492-9d31-6615cde245d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '054d9c01cc80408084cbfe08418ef917'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:23.762352 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3f59f3c0-8214-4492-9d31-6615cde245d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '054d9c01cc80408084cbfe08418ef917'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:23.763351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3f59f3c0-8214-4492-9d31-6615cde245d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '054d9c01cc80408084cbfe08418ef917'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:23.780337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1085/2196] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:23 2022] DELETE /identity/v3/users/054d9c01cc80408084cbfe08418ef917 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:49:23.781806 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1112/2197] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:23 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:23.784896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59f3c0-8214-4492-9d31-6615cde245d5 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:23.787016 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3f59f3c0-8214-4492-9d31-6615cde245d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:23.831510 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:23.857478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59f3c0-8214-4492-9d31-6615cde245d5 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:23.862465 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b623e-56b6-42da-bbeb-76c7ace6dbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:23.863148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b623e-56b6-42da-bbeb-76c7ace6dbbf admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:23.863431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b623e-56b6-42da-bbeb-76c7ace6dbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:23.863706 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b623e-56b6-42da-bbeb-76c7ace6dbbf admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:23.865053 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b623e-56b6-42da-bbeb-76c7ace6dbbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:23.867380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b623e-56b6-42da-bbeb-76c7ace6dbbf admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:23.889454 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1086/2198] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:23 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:23.893198 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc2156-03f4-4f1d-b077-c146b7fd0b1a admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:23.962370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc2156-03f4-4f1d-b077-c146b7fd0b1a admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:23.967765 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d78eec-8736-4635-828e-925d6e5d7cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:23.968465 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d78eec-8736-4635-828e-925d6e5d7cdf admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:23.968733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d78eec-8736-4635-828e-925d6e5d7cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:23.969016 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d78eec-8736-4635-828e-925d6e5d7cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:23.969835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d78eec-8736-4635-828e-925d6e5d7cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:23.971944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d78eec-8736-4635-828e-925d6e5d7cdf admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:23.983354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1113/2199] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:23.987338 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b623e-56b6-42da-bbeb-76c7ace6dbbf admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:24.007200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b623e-56b6-42da-bbeb-76c7ace6dbbf admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:24.012029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e885d8-6449-439e-a467-a6cf3286e5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:24.012707 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e885d8-6449-439e-a467-a6cf3286e5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:24.012987 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e885d8-6449-439e-a467-a6cf3286e5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:24.013262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e885d8-6449-439e-a467-a6cf3286e5d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:24.014046 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e885d8-6449-439e-a467-a6cf3286e5d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:24.015639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e885d8-6449-439e-a467-a6cf3286e5d7 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:24.027009 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1087/2200] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:24.031146 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d78eec-8736-4635-828e-925d6e5d7cdf admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:24.042802 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d78eec-8736-4635-828e-925d6e5d7cdf admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:24.048342 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5bc58-69b3-4f98-bc8f-614ebb96895d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:24.049323 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5bc58-69b3-4f98-bc8f-614ebb96895d admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:24.049650 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5bc58-69b3-4f98-bc8f-614ebb96895d admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:24.049926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5bc58-69b3-4f98-bc8f-614ebb96895d admin admin] PATH_INFO: `/v3/projects/3c6536154b424b40abb6a19fef51867c/users/eb2f0ac0e17448f58265379e0ea53b19/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:24.065903 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5bc58-69b3-4f98-bc8f-614ebb96895d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c6536154b424b40abb6a19fef51867c, user_id=eb2f0ac0e17448f58265379e0ea53b19, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:24.069310 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5bc58-69b3-4f98-bc8f-614ebb96895d admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:24.085682 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1114/2201] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:24 2022] PUT /identity/v3/projects/3c6536154b424b40abb6a19fef51867c/users/eb2f0ac0e17448f58265379e0ea53b19/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:24.095965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d216496-cfdd-47f8-8428-94cb7fffe464 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:24.096262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d216496-cfdd-47f8-8428-94cb7fffe464 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:24.096546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d216496-cfdd-47f8-8428-94cb7fffe464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:24.127201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-7d216496-cfdd-47f8-8428-94cb7fffe464 None None] MFA Rules not processed for user `eb2f0ac0e17448f58265379e0ea53b19`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:24.128451 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-7d216496-cfdd-47f8-8428-94cb7fffe464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:24.212959 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1088/2202] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:24.226701 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5bc58-69b3-4f98-bc8f-614ebb96895d admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:24.228679 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-a0c5bc58-69b3-4f98-bc8f-614ebb96895d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:24.312537 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5bc58-69b3-4f98-bc8f-614ebb96895d admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:24.319131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9529653-c250-494d-b3c8-d3c57d844826 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493fcde4a45944efbc0b52c44e61b411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:24.320073 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9529653-c250-494d-b3c8-d3c57d844826 service glance] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:24.320366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9529653-c250-494d-b3c8-d3c57d844826 service glance] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:24.320647 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9529653-c250-494d-b3c8-d3c57d844826 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:24.344770 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9529653-c250-494d-b3c8-d3c57d844826 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:24.345414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d216496-cfdd-47f8-8428-94cb7fffe464 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:24.346314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9529653-c250-494d-b3c8-d3c57d844826 service glance] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:24.346991 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-7d216496-cfdd-47f8-8428-94cb7fffe464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:24.361349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1115/2203] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:24.442847 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9529653-c250-494d-b3c8-d3c57d844826 service glance] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:24.448458 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e9529653-c250-494d-b3c8-d3c57d844826 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:24.464659 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d216496-cfdd-47f8-8428-94cb7fffe464 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:24.472285 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445d235-3914-45ce-9be5-94d5062618a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:24.473178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445d235-3914-45ce-9be5-94d5062618a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:24.474315 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445d235-3914-45ce-9be5-94d5062618a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:24.474652 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445d235-3914-45ce-9be5-94d5062618a0 admin admin] PATH_INFO: `/v3/projects/a552ac26c5c14159a1c62889ead232c7` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:24.475920 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445d235-3914-45ce-9be5-94d5062618a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a552ac26c5c14159a1c62889ead232c7)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:24.479613 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445d235-3914-45ce-9be5-94d5062618a0 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:24.511915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9529653-c250-494d-b3c8-d3c57d844826 service glance] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:24.538439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3a9f7-8759-4346-8671-16c68468d6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:24.539417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3a9f7-8759-4346-8671-16c68468d6e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:24.539721 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3a9f7-8759-4346-8671-16c68468d6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:24.540053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3a9f7-8759-4346-8671-16c68468d6e2 admin admin] PATH_INFO: `/v3/users/91ad7f548b55489a80c395dfaa0efa35` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:24.541569 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3a9f7-8759-4346-8671-16c68468d6e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ad7f548b55489a80c395dfaa0efa35)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:24.544343 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3a9f7-8759-4346-8671-16c68468d6e2 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:24.581421 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:24.581978 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:24.586038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f445d235-3914-45ce-9be5-94d5062618a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a552ac26c5c14159a1c62889ead232c7', 'request_id': 'req-f445d235-3914-45ce-9be5-94d5062618a0'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:24.600391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f445d235-3914-45ce-9be5-94d5062618a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a552ac26c5c14159a1c62889ead232c7', 'request_id': 'req-f445d235-3914-45ce-9be5-94d5062618a0'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:24.609079 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f445d235-3914-45ce-9be5-94d5062618a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a552ac26c5c14159a1c62889ead232c7', 'request_id': 'req-f445d235-3914-45ce-9be5-94d5062618a0'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:24.628244 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f445d235-3914-45ce-9be5-94d5062618a0 admin admin] The token cache is being invalidate because project a552ac26c5c14159a1c62889ead232c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:24.628666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f445d235-3914-45ce-9be5-94d5062618a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:24.629602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f445d235-3914-45ce-9be5-94d5062618a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:24.633132 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1089/2204] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:24 2022] DELETE /identity/v3/projects/a552ac26c5c14159a1c62889ead232c7 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:49:24.638176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebeea1-d4fa-463f-afae-cdd874fe0a36 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:24.638576 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebeea1-d4fa-463f-afae-cdd874fe0a36 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:24.639083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebeea1-d4fa-463f-afae-cdd874fe0a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:24.639537 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8fb3a9f7-8759-4346-8671-16c68468d6e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ad7f548b55489a80c395dfaa0efa35'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:24.659009 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8fb3a9f7-8759-4346-8671-16c68468d6e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ad7f548b55489a80c395dfaa0efa35'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:24.659774 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8fb3a9f7-8759-4346-8671-16c68468d6e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ad7f548b55489a80c395dfaa0efa35'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:24.670001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-69ebeea1-d4fa-463f-afae-cdd874fe0a36 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:24.671895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-69ebeea1-d4fa-463f-afae-cdd874fe0a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:24.687677 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8fb3a9f7-8759-4346-8671-16c68468d6e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ad7f548b55489a80c395dfaa0efa35'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:24.700278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8fb3a9f7-8759-4346-8671-16c68468d6e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ad7f548b55489a80c395dfaa0efa35'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:24.709121 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1116/2205] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:24 2022] DELETE /identity/v3/users/91ad7f548b55489a80c395dfaa0efa35 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:24.713629 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25baf2-0b38-4408-a376-23c112be4c22 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:24.714055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25baf2-0b38-4408-a376-23c112be4c22 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:24.715075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25baf2-0b38-4408-a376-23c112be4c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:24.741378 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-8c25baf2-0b38-4408-a376-23c112be4c22 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:24.742674 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8c25baf2-0b38-4408-a376-23c112be4c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:24.766074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1090/2206] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:24.779232 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175209e7-21f9-4de6-8aa1-81a2d57ffb41 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:24.779730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175209e7-21f9-4de6-8aa1-81a2d57ffb41 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:24.781949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175209e7-21f9-4de6-8aa1-81a2d57ffb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:24.807555 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-175209e7-21f9-4de6-8aa1-81a2d57ffb41 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:24.809153 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-175209e7-21f9-4de6-8aa1-81a2d57ffb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:24.872702 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1117/2207] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:24.885235 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25baf2-0b38-4408-a376-23c112be4c22 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:24.888482 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8c25baf2-0b38-4408-a376-23c112be4c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:24.921167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1091/2208] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:24.925358 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175209e7-21f9-4de6-8aa1-81a2d57ffb41 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:24.945720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175209e7-21f9-4de6-8aa1-81a2d57ffb41 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:24.972082 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b0990-4e1d-47a2-b3f9-cfa034225f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:24.972805 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b0990-4e1d-47a2-b3f9-cfa034225f47 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:24.973093 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b0990-4e1d-47a2-b3f9-cfa034225f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:24.973527 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b0990-4e1d-47a2-b3f9-cfa034225f47 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:24.974926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b0990-4e1d-47a2-b3f9-cfa034225f47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:24.975248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b0990-4e1d-47a2-b3f9-cfa034225f47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:24.977288 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b0990-4e1d-47a2-b3f9-cfa034225f47 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:25.003365 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1092/2209] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:49:25.009483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b0990-4e1d-47a2-b3f9-cfa034225f47 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:25.031130 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25baf2-0b38-4408-a376-23c112be4c22 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:25.036228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6aa8a-3d99-410e-95d4-fc5099e3a6cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:25.036911 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6aa8a-3d99-410e-95d4-fc5099e3a6cf service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:25.037135 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b0990-4e1d-47a2-b3f9-cfa034225f47 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:25.037265 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6aa8a-3d99-410e-95d4-fc5099e3a6cf service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:25.037443 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6aa8a-3d99-410e-95d4-fc5099e3a6cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:25.052427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f187a-25d0-4318-a21c-3dd1395b693c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:25.054739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f187a-25d0-4318-a21c-3dd1395b693c admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:25.055128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6aa8a-3d99-410e-95d4-fc5099e3a6cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:25.055783 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f187a-25d0-4318-a21c-3dd1395b693c admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:25.056690 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6aa8a-3d99-410e-95d4-fc5099e3a6cf service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:25.057767 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f187a-25d0-4318-a21c-3dd1395b693c admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:25.058688 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f187a-25d0-4318-a21c-3dd1395b693c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:25.064677 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f187a-25d0-4318-a21c-3dd1395b693c admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:25.071453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1118/2210] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:49:25.160597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1093/2211] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:25 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:25.165694 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6aa8a-3d99-410e-95d4-fc5099e3a6cf service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:25.180169 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6aa8a-3d99-410e-95d4-fc5099e3a6cf service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:25.252864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:25.314158 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d2c1f-a96c-41e4-a8ee-114f66663bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:25.315109 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d2c1f-a96c-41e4-a8ee-114f66663bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:25.315491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d2c1f-a96c-41e4-a8ee-114f66663bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:25.315717 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d2c1f-a96c-41e4-a8ee-114f66663bf9 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:25.316460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d2c1f-a96c-41e4-a8ee-114f66663bf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:25.319012 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635d2c1f-a96c-41e4-a8ee-114f66663bf9 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:25.346377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1119/2212] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:25 2022] POST /identity/v3/users => generated 292 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:25.353443 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f187a-25d0-4318-a21c-3dd1395b693c admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:25.368280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f187a-25d0-4318-a21c-3dd1395b693c admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:25.374596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d698fa-2117-4630-911b-711c83af8f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:25.375034 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d698fa-2117-4630-911b-711c83af8f19 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:25.375327 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d698fa-2117-4630-911b-711c83af8f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:25.375965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d698fa-2117-4630-911b-711c83af8f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:25.376609 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d698fa-2117-4630-911b-711c83af8f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:25.378297 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d698fa-2117-4630-911b-711c83af8f19 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:25.390832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1094/2213] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:25.395531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d2c1f-a96c-41e4-a8ee-114f66663bf9 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:25.406230 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d2c1f-a96c-41e4-a8ee-114f66663bf9 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:25.411627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be65b9-e75b-4895-8cf8-6e48364538c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:25.412316 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be65b9-e75b-4895-8cf8-6e48364538c5 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:25.412602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be65b9-e75b-4895-8cf8-6e48364538c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:25.412880 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be65b9-e75b-4895-8cf8-6e48364538c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:25.413629 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be65b9-e75b-4895-8cf8-6e48364538c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:25.415356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be65b9-e75b-4895-8cf8-6e48364538c5 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:25.427425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1120/2214] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:25.432223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d698fa-2117-4630-911b-711c83af8f19 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:25.445387 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d698fa-2117-4630-911b-711c83af8f19 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:25.451763 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e97736-6184-434f-80e7-c6fddf6731b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:25.453556 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e97736-6184-434f-80e7-c6fddf6731b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:25.453896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e97736-6184-434f-80e7-c6fddf6731b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:25.454250 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e97736-6184-434f-80e7-c6fddf6731b8 admin admin] PATH_INFO: `/v3/projects/e552ed97892f41259a1d7057aea64f6b/users/7432931bd04d45bcaee686df05d5c5b3/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:25.477816 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e97736-6184-434f-80e7-c6fddf6731b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e552ed97892f41259a1d7057aea64f6b, user_id=7432931bd04d45bcaee686df05d5c5b3, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:25.480642 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e97736-6184-434f-80e7-c6fddf6731b8 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:25.493738 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be65b9-e75b-4895-8cf8-6e48364538c5 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:25.496427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-48be65b9-e75b-4895-8cf8-6e48364538c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:25.503189 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1095/2215] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:25 2022] PUT /identity/v3/projects/e552ed97892f41259a1d7057aea64f6b/users/7432931bd04d45bcaee686df05d5c5b3/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:25.509981 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700dfd6-6474-4628-b51c-fbdd2c23883f None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:25.511967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700dfd6-6474-4628-b51c-fbdd2c23883f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:25.512254 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700dfd6-6474-4628-b51c-fbdd2c23883f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:25.517384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be65b9-e75b-4895-8cf8-6e48364538c5 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:25.540810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:25.558999 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-5700dfd6-6474-4628-b51c-fbdd2c23883f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:25.560363 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-5700dfd6-6474-4628-b51c-fbdd2c23883f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:25.604038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d57f1-8d41-40d0-9e4d-f137a4263646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:25.604974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d57f1-8d41-40d0-9e4d-f137a4263646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:25.605384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d57f1-8d41-40d0-9e4d-f137a4263646 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:25.605822 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d57f1-8d41-40d0-9e4d-f137a4263646 admin admin] PATH_INFO: `/v3/projects/72fff1ebf5e044b09b04b80454c87d57` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:25.607161 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d57f1-8d41-40d0-9e4d-f137a4263646 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72fff1ebf5e044b09b04b80454c87d57)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:25.609343 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d57f1-8d41-40d0-9e4d-f137a4263646 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:25.633985 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1096/2216] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:25.643631 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700dfd6-6474-4628-b51c-fbdd2c23883f None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:25.672020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-2e9d57f1-8d41-40d0-9e4d-f137a4263646 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72fff1ebf5e044b09b04b80454c87d57', 'request_id': 'req-2e9d57f1-8d41-40d0-9e4d-f137a4263646'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:25.683609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-2e9d57f1-8d41-40d0-9e4d-f137a4263646 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72fff1ebf5e044b09b04b80454c87d57', 'request_id': 'req-2e9d57f1-8d41-40d0-9e4d-f137a4263646'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:25.693169 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-2e9d57f1-8d41-40d0-9e4d-f137a4263646 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72fff1ebf5e044b09b04b80454c87d57', 'request_id': 'req-2e9d57f1-8d41-40d0-9e4d-f137a4263646'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:25.709062 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-2e9d57f1-8d41-40d0-9e4d-f137a4263646 admin admin] The token cache is being invalidate because project 72fff1ebf5e044b09b04b80454c87d57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:25.709616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-2e9d57f1-8d41-40d0-9e4d-f137a4263646 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:25.710626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-2e9d57f1-8d41-40d0-9e4d-f137a4263646 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:25.712870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1121/2217] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:25 2022] DELETE /identity/v3/projects/72fff1ebf5e044b09b04b80454c87d57 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:25.716830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d57f1-8d41-40d0-9e4d-f137a4263646 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:25.718238 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2e9d57f1-8d41-40d0-9e4d-f137a4263646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:25.725739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700dfd6-6474-4628-b51c-fbdd2c23883f None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:25.814617 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fba021-2ee2-486c-ad3d-f9e825d37037 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:25.814995 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fba021-2ee2-486c-ad3d-f9e825d37037 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:25.815417 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fba021-2ee2-486c-ad3d-f9e825d37037 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:25.816370 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fba021-2ee2-486c-ad3d-f9e825d37037 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:25.818252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-c9fba021-2ee2-486c-ad3d-f9e825d37037 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:25.831911 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:25.882535 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d57f1-8d41-40d0-9e4d-f137a4263646 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:25.892998 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa5c58-e41e-4771-937e-917d9dc30220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:25.893929 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa5c58-e41e-4771-937e-917d9dc30220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:25.894216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa5c58-e41e-4771-937e-917d9dc30220 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:25.894498 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa5c58-e41e-4771-937e-917d9dc30220 admin admin] PATH_INFO: `/v3/users/5af4b3ba6af143a496939d1e620a9b3d` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:25.896152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa5c58-e41e-4771-937e-917d9dc30220 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5af4b3ba6af143a496939d1e620a9b3d)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:25.901186 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa5c58-e41e-4771-937e-917d9dc30220 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:25.948358 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fba021-2ee2-486c-ad3d-f9e825d37037 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:25.949831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fba021-2ee2-486c-ad3d-f9e825d37037 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:25.961858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1097/2218] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:49:25.979693 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8daa5c58-e41e-4771-937e-917d9dc30220 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5af4b3ba6af143a496939d1e620a9b3d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:26.009378 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8daa5c58-e41e-4771-937e-917d9dc30220 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5af4b3ba6af143a496939d1e620a9b3d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:26.010845 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8daa5c58-e41e-4771-937e-917d9dc30220 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5af4b3ba6af143a496939d1e620a9b3d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:26.024103 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8daa5c58-e41e-4771-937e-917d9dc30220 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5af4b3ba6af143a496939d1e620a9b3d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:26.046041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8daa5c58-e41e-4771-937e-917d9dc30220 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5af4b3ba6af143a496939d1e620a9b3d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:26.070374 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1122/2219] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:25 2022] DELETE /identity/v3/users/5af4b3ba6af143a496939d1e620a9b3d => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:49:26.803857 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809d53a-859c-4266-911f-10262a3da445 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:26.804469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809d53a-859c-4266-911f-10262a3da445 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:26.804469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809d53a-859c-4266-911f-10262a3da445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:26.853676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:26.856135 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-1809d53a-859c-4266-911f-10262a3da445 None None] MFA Rules not processed for user `7432931bd04d45bcaee686df05d5c5b3`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:26.857966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1809d53a-859c-4266-911f-10262a3da445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:27.004377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1098/2220] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:27.029080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa5c58-e41e-4771-937e-917d9dc30220 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:27.030641 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8daa5c58-e41e-4771-937e-917d9dc30220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:27.124116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa5c58-e41e-4771-937e-917d9dc30220 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:27.140167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9e7b8-df21-4dff-8ea3-32373446ec48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:27.141627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9e7b8-df21-4dff-8ea3-32373446ec48 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:27.142251 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9e7b8-df21-4dff-8ea3-32373446ec48 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:27.142626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9e7b8-df21-4dff-8ea3-32373446ec48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:27.156193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd9e7b8-df21-4dff-8ea3-32373446ec48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:27.160627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd9e7b8-df21-4dff-8ea3-32373446ec48 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:27.193212 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1123/2221] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:27.248633 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809d53a-859c-4266-911f-10262a3da445 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:27.253340 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1809d53a-859c-4266-911f-10262a3da445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:27.396858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809d53a-859c-4266-911f-10262a3da445 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:27.422313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7c985-6d67-404f-9516-3ef6ac1b4d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:27.423939 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7c985-6d67-404f-9516-3ef6ac1b4d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:27.424699 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7c985-6d67-404f-9516-3ef6ac1b4d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:27.425677 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7c985-6d67-404f-9516-3ef6ac1b4d3d admin admin] PATH_INFO: `/v3/projects/971b6e0af6014bf68ad1e6fe76e02380` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:27.427962 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7c985-6d67-404f-9516-3ef6ac1b4d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=971b6e0af6014bf68ad1e6fe76e02380)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:27.430280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7c985-6d67-404f-9516-3ef6ac1b4d3d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:27.505674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-13f7c985-6d67-404f-9516-3ef6ac1b4d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '971b6e0af6014bf68ad1e6fe76e02380', 'request_id': 'req-13f7c985-6d67-404f-9516-3ef6ac1b4d3d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:27.522798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-13f7c985-6d67-404f-9516-3ef6ac1b4d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '971b6e0af6014bf68ad1e6fe76e02380', 'request_id': 'req-13f7c985-6d67-404f-9516-3ef6ac1b4d3d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:27.533195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-13f7c985-6d67-404f-9516-3ef6ac1b4d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '971b6e0af6014bf68ad1e6fe76e02380', 'request_id': 'req-13f7c985-6d67-404f-9516-3ef6ac1b4d3d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:27.559206 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-13f7c985-6d67-404f-9516-3ef6ac1b4d3d admin admin] The token cache is being invalidate because project 971b6e0af6014bf68ad1e6fe76e02380 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:27.560286 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-13f7c985-6d67-404f-9516-3ef6ac1b4d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:27.561606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-13f7c985-6d67-404f-9516-3ef6ac1b4d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:27.569386 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1099/2222] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:27 2022] DELETE /identity/v3/projects/971b6e0af6014bf68ad1e6fe76e02380 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:27.570446 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8a170-f05e-4b5d-8765-276a3de4cf82 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:27.570767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8a170-f05e-4b5d-8765-276a3de4cf82 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:27.571054 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8a170-f05e-4b5d-8765-276a3de4cf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:27.609224 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-3fa8a170-f05e-4b5d-8765-276a3de4cf82 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:27.611031 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-3fa8a170-f05e-4b5d-8765-276a3de4cf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:27.776240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1124/2223] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:27.786930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4964c29-86fb-4f6d-8603-0246e0eef881 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:27.787229 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4964c29-86fb-4f6d-8603-0246e0eef881 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:27.787515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4964c29-86fb-4f6d-8603-0246e0eef881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:27.819819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-c4964c29-86fb-4f6d-8603-0246e0eef881 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:27.821067 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-c4964c29-86fb-4f6d-8603-0246e0eef881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:27.838094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1100/2224] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:27.842781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8a170-f05e-4b5d-8765-276a3de4cf82 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:27.860139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8a170-f05e-4b5d-8765-276a3de4cf82 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:27.875840 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ffe35-1705-4cbe-8162-a53bf14351a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:27.876918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ffe35-1705-4cbe-8162-a53bf14351a8 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:27.877210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ffe35-1705-4cbe-8162-a53bf14351a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:27.877593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ffe35-1705-4cbe-8162-a53bf14351a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:27.878544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ffe35-1705-4cbe-8162-a53bf14351a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:27.878977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ffe35-1705-4cbe-8162-a53bf14351a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:27.880833 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ffe35-1705-4cbe-8162-a53bf14351a8 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:27.882827 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:27.901575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4964c29-86fb-4f6d-8603-0246e0eef881 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:27.905851 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1125/2225] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:49:27.912361 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4964c29-86fb-4f6d-8603-0246e0eef881 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:27.917261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1e615-03d7-44bc-80a5-0ee92e1164af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:27.917983 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1e615-03d7-44bc-80a5-0ee92e1164af admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:27.918267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1e615-03d7-44bc-80a5-0ee92e1164af admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:27.918543 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1e615-03d7-44bc-80a5-0ee92e1164af admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:27.919243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1e615-03d7-44bc-80a5-0ee92e1164af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:27.921687 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1e615-03d7-44bc-80a5-0ee92e1164af admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:27.958849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1101/2226] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:27 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:27.961039 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ffe35-1705-4cbe-8162-a53bf14351a8 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:27.972067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ffe35-1705-4cbe-8162-a53bf14351a8 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:28.052261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba63d35-af27-44bd-923a-29aa79dae168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:28.053156 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba63d35-af27-44bd-923a-29aa79dae168 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:28.053584 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba63d35-af27-44bd-923a-29aa79dae168 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:28.053962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba63d35-af27-44bd-923a-29aa79dae168 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:28.054828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba63d35-af27-44bd-923a-29aa79dae168 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:28.057216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba63d35-af27-44bd-923a-29aa79dae168 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:28.094162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1e615-03d7-44bc-80a5-0ee92e1164af admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:28.094587 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1126/2227] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:27 2022] POST /identity/v3/users => generated 300 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:28.101837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:28.107251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1e615-03d7-44bc-80a5-0ee92e1164af admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:28.120185 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a3e47-9304-46b3-904f-3eff3bfbd43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:28.121016 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a3e47-9304-46b3-904f-3eff3bfbd43d admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:28.121366 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a3e47-9304-46b3-904f-3eff3bfbd43d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:28.121943 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a3e47-9304-46b3-904f-3eff3bfbd43d admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:28.123340 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a3e47-9304-46b3-904f-3eff3bfbd43d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:28.124924 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a3e47-9304-46b3-904f-3eff3bfbd43d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:28.146618 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba63d35-af27-44bd-923a-29aa79dae168 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:28.162275 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1102/2228] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:28.171661 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba63d35-af27-44bd-923a-29aa79dae168 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:28.209317 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4513e-c01c-4f51-98e6-2a30356b6c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:28.210129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4513e-c01c-4f51-98e6-2a30356b6c26 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:28.210414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4513e-c01c-4f51-98e6-2a30356b6c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:28.210762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4513e-c01c-4f51-98e6-2a30356b6c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:28.211551 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4513e-c01c-4f51-98e6-2a30356b6c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:28.213218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4513e-c01c-4f51-98e6-2a30356b6c26 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:28.251941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1127/2229] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:28.258725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a3e47-9304-46b3-904f-3eff3bfbd43d admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:28.274540 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a3e47-9304-46b3-904f-3eff3bfbd43d admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:28.280597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c21799-229d-48ca-bc13-f4454a3e77e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:28.282035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c21799-229d-48ca-bc13-f4454a3e77e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:28.282543 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c21799-229d-48ca-bc13-f4454a3e77e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:28.282967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c21799-229d-48ca-bc13-f4454a3e77e7 admin admin] PATH_INFO: `/v3/projects/c630467b763a410e936008052e5fdb08/users/b22e385b694c4a35a7727c4f8d4a080d/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:28.336918 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c21799-229d-48ca-bc13-f4454a3e77e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c630467b763a410e936008052e5fdb08, user_id=b22e385b694c4a35a7727c4f8d4a080d, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:28.341547 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c21799-229d-48ca-bc13-f4454a3e77e7 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:28.374857 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1103/2230] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:28 2022] PUT /identity/v3/projects/c630467b763a410e936008052e5fdb08/users/b22e385b694c4a35a7727c4f8d4a080d/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:28.389154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968b0e1-acb5-49c7-b921-1b7068b6c276 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:28.389154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968b0e1-acb5-49c7-b921-1b7068b6c276 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:28.389154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968b0e1-acb5-49c7-b921-1b7068b6c276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:28.470369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-5968b0e1-acb5-49c7-b921-1b7068b6c276 None None] MFA Rules not processed for user `b22e385b694c4a35a7727c4f8d4a080d`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:28.471654 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-5968b0e1-acb5-49c7-b921-1b7068b6c276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:28.585876 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1128/2231] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:28.586259 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c21799-229d-48ca-bc13-f4454a3e77e7 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:28.662976 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c21799-229d-48ca-bc13-f4454a3e77e7 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:28.676984 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73673858-0fdb-457d-a5a9-da610a1b99e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:28.678025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73673858-0fdb-457d-a5a9-da610a1b99e1 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:28.678410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73673858-0fdb-457d-a5a9-da610a1b99e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:28.678756 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73673858-0fdb-457d-a5a9-da610a1b99e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:28.679566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73673858-0fdb-457d-a5a9-da610a1b99e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:28.681661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73673858-0fdb-457d-a5a9-da610a1b99e1 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:28.722792 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1104/2232] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:28 2022] POST /identity/v3/projects => generated 399 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:49:28.725348 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968b0e1-acb5-49c7-b921-1b7068b6c276 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:28.737294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968b0e1-acb5-49c7-b921-1b7068b6c276 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:28.812210 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be8882-cf9b-40b3-b30e-40a605cab485 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:28.813320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be8882-cf9b-40b3-b30e-40a605cab485 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:28.813320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be8882-cf9b-40b3-b30e-40a605cab485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:28.851776 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-86be8882-cf9b-40b3-b30e-40a605cab485 None None] MFA Rules not processed for user `7432931bd04d45bcaee686df05d5c5b3`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:28.853130 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-86be8882-cf9b-40b3-b30e-40a605cab485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:28.870493 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10ec44-bc7a-4f90-9039-226866bc54bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:28.871786 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10ec44-bc7a-4f90-9039-226866bc54bd admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:28.872968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10ec44-bc7a-4f90-9039-226866bc54bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:28.874309 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10ec44-bc7a-4f90-9039-226866bc54bd admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:28.875294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10ec44-bc7a-4f90-9039-226866bc54bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:28.878087 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10ec44-bc7a-4f90-9039-226866bc54bd admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:28.889447 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:28.914292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1129/2233] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:28 2022] POST /identity/v3/users => generated 306 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:28.918756 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10ec44-bc7a-4f90-9039-226866bc54bd admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:28.932399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10ec44-bc7a-4f90-9039-226866bc54bd admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:28.945551 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b866db86-0099-4a6b-b461-3f9cf0760794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:28.946413 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866db86-0099-4a6b-b461-3f9cf0760794 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:28.946751 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866db86-0099-4a6b-b461-3f9cf0760794 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:28.947089 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866db86-0099-4a6b-b461-3f9cf0760794 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:28.948417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b866db86-0099-4a6b-b461-3f9cf0760794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:28.951441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b866db86-0099-4a6b-b461-3f9cf0760794 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:28.969703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1130/2234] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:28.974721 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b866db86-0099-4a6b-b461-3f9cf0760794 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:28.987919 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b866db86-0099-4a6b-b461-3f9cf0760794 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:28.994988 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e3062-aaa6-4e56-8fc9-8f35808b21c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:28.996093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e3062-aaa6-4e56-8fc9-8f35808b21c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:28.996476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e3062-aaa6-4e56-8fc9-8f35808b21c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:28.996849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e3062-aaa6-4e56-8fc9-8f35808b21c9 admin admin] PATH_INFO: `/v3/projects/2e4f8598bff54195ad246fde2b50fe28/users/91875588184340eda0e8ba0c449ee49c/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:29.025409 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e3062-aaa6-4e56-8fc9-8f35808b21c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e4f8598bff54195ad246fde2b50fe28, user_id=91875588184340eda0e8ba0c449ee49c, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:29.028468 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e3062-aaa6-4e56-8fc9-8f35808b21c9 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:29.065711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1105/2235] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:29.065891 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1131/2236] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:28 2022] PUT /identity/v3/projects/2e4f8598bff54195ad246fde2b50fe28/users/91875588184340eda0e8ba0c449ee49c/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:29.074688 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e3062-aaa6-4e56-8fc9-8f35808b21c9 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:29.130282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90043949-f873-4806-a803-661ead93a3f7 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:29.130527 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90043949-f873-4806-a803-661ead93a3f7 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:29.130796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90043949-f873-4806-a803-661ead93a3f7 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:29.132162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1106/2237] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jun 27 15:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:49:29.183311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4a891-6304-4b9b-9882-5f272f615d4e None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:29.183595 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4a891-6304-4b9b-9882-5f272f615d4e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:29.183879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4a891-6304-4b9b-9882-5f272f615d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:29.214019 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e3062-aaa6-4e56-8fc9-8f35808b21c9 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:29.220931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720c080-261f-43dc-b183-e6365786a1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:29.221768 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720c080-261f-43dc-b183-e6365786a1c0 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:29.222158 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720c080-261f-43dc-b183-e6365786a1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:29.222538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720c080-261f-43dc-b183-e6365786a1c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:29.224244 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720c080-261f-43dc-b183-e6365786a1c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:29.226336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720c080-261f-43dc-b183-e6365786a1c0 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:29.226601 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-09d4a891-6304-4b9b-9882-5f272f615d4e None None] MFA Rules not processed for user `ffafb68f1bb941588a5217ad83ad907c`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:29.228068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-09d4a891-6304-4b9b-9882-5f272f615d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:29.245607 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1132/2238] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:29.248466 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720c080-261f-43dc-b183-e6365786a1c0 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:29.264553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720c080-261f-43dc-b183-e6365786a1c0 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:29.283901 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15753319-286f-49d8-9acd-b0f3e0adce37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:29.284638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15753319-286f-49d8-9acd-b0f3e0adce37 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:29.284914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15753319-286f-49d8-9acd-b0f3e0adce37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:29.285199 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15753319-286f-49d8-9acd-b0f3e0adce37 admin admin] PATH_INFO: `/v3/projects/2e4f8598bff54195ad246fde2b50fe28/users/91875588184340eda0e8ba0c449ee49c/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:29.289408 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15753319-286f-49d8-9acd-b0f3e0adce37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e4f8598bff54195ad246fde2b50fe28, user_id=91875588184340eda0e8ba0c449ee49c, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:29.292978 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15753319-286f-49d8-9acd-b0f3e0adce37 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:29.307186 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1133/2239] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:29 2022] PUT /identity/v3/projects/2e4f8598bff54195ad246fde2b50fe28/users/91875588184340eda0e8ba0c449ee49c/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:29.319191 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d11d4-5c70-43d4-991d-f53fa9f20e13 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:29.319523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d11d4-5c70-43d4-991d-f53fa9f20e13 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:29.319762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d11d4-5c70-43d4-991d-f53fa9f20e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:29.331073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:29.366690 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1107/2240] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jun 27 15:49:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:29.370962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-5d9d11d4-5c70-43d4-991d-f53fa9f20e13 None None] MFA Rules not processed for user `91875588184340eda0e8ba0c449ee49c`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:29.375139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-5d9d11d4-5c70-43d4-991d-f53fa9f20e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:29.378167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4a891-6304-4b9b-9882-5f272f615d4e None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:29.380945 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-09d4a891-6304-4b9b-9882-5f272f615d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:29.403901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4a891-6304-4b9b-9882-5f272f615d4e None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:29.497901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe86b0-dd55-4d43-a3e6-f4ffff953dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:29.499891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe86b0-dd55-4d43-a3e6-f4ffff953dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:29.499891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe86b0-dd55-4d43-a3e6-f4ffff953dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:29.499891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe86b0-dd55-4d43-a3e6-f4ffff953dd2 admin admin] PATH_INFO: `/v3/users/3520bd4387a140f88558d5f6fe55986e` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:29.500849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe86b0-dd55-4d43-a3e6-f4ffff953dd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3520bd4387a140f88558d5f6fe55986e)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:29.502938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe86b0-dd55-4d43-a3e6-f4ffff953dd2 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:29.518531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1134/2241] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:29 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:29.519547 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d11d4-5c70-43d4-991d-f53fa9f20e13 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:29.525373 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-5d9d11d4-5c70-43d4-991d-f53fa9f20e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:29.573338 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:29.589492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-5fbe86b0-dd55-4d43-a3e6-f4ffff953dd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3520bd4387a140f88558d5f6fe55986e'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:29.614609 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-5fbe86b0-dd55-4d43-a3e6-f4ffff953dd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3520bd4387a140f88558d5f6fe55986e'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:29.625545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-5fbe86b0-dd55-4d43-a3e6-f4ffff953dd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3520bd4387a140f88558d5f6fe55986e'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:29.626416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-5fbe86b0-dd55-4d43-a3e6-f4ffff953dd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3520bd4387a140f88558d5f6fe55986e'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:29.627153 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-5fbe86b0-dd55-4d43-a3e6-f4ffff953dd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3520bd4387a140f88558d5f6fe55986e'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:29.645884 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1108/2242] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:29 2022] DELETE /identity/v3/users/3520bd4387a140f88558d5f6fe55986e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:29.649433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe86b0-dd55-4d43-a3e6-f4ffff953dd2 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:29.653480 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-5fbe86b0-dd55-4d43-a3e6-f4ffff953dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:29.657570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d11d4-5c70-43d4-991d-f53fa9f20e13 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:29.742903 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe86b0-dd55-4d43-a3e6-f4ffff953dd2 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:29.750053 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42a8f0-5c3c-442a-84c3-1c4d61b5a8fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:29.750822 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42a8f0-5c3c-442a-84c3-1c4d61b5a8fe service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:29.751251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42a8f0-5c3c-442a-84c3-1c4d61b5a8fe service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:29.751630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42a8f0-5c3c-442a-84c3-1c4d61b5a8fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:29.754026 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-cc42a8f0-5c3c-442a-84c3-1c4d61b5a8fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:29.758120 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ffbf8-959a-47ee-bbff-61f3c903d221 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffafb68f1bb941588a5217ad83ad907c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:29.758797 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ffbf8-959a-47ee-bbff-61f3c903d221 service placement] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:29.759172 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ffbf8-959a-47ee-bbff-61f3c903d221 service placement] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:29.759467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ffbf8-959a-47ee-bbff-61f3c903d221 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:29.761994 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-5c8ffbf8-959a-47ee-bbff-61f3c903d221 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:29.777005 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ffbf8-959a-47ee-bbff-61f3c903d221 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:29.779082 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ffbf8-959a-47ee-bbff-61f3c903d221 service placement] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:29.794228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1135/2243] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:29.807498 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc40c51-852e-4696-a2cc-9dc4691b76e0 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:29.808441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc40c51-852e-4696-a2cc-9dc4691b76e0 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:29.809609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc40c51-852e-4696-a2cc-9dc4691b76e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:29.812412 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:29.849505 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-3bc40c51-852e-4696-a2cc-9dc4691b76e0 None None] MFA Rules not processed for user `6655366e7eb84a26aa3f7fbe88af62b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:29.850897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-3bc40c51-852e-4696-a2cc-9dc4691b76e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:29.890536 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42a8f0-5c3c-442a-84c3-1c4d61b5a8fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:29.894061 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42a8f0-5c3c-442a-84c3-1c4d61b5a8fe service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:29.906169 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1109/2244] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:29 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:29.910258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42a8f0-5c3c-442a-84c3-1c4d61b5a8fe service cinder] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:29.912705 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-cc42a8f0-5c3c-442a-84c3-1c4d61b5a8fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:29.978758 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1136/2245] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jun 27 15:49:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:30.063331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc40c51-852e-4696-a2cc-9dc4691b76e0 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:30.067620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-3bc40c51-852e-4696-a2cc-9dc4691b76e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:30.104068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42a8f0-5c3c-442a-84c3-1c4d61b5a8fe service cinder] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:30.109890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1218f-b053-4c6c-9b30-2656578d5c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:30.110894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1218f-b053-4c6c-9b30-2656578d5c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:30.111320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1218f-b053-4c6c-9b30-2656578d5c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:30.111707 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1218f-b053-4c6c-9b30-2656578d5c6f admin admin] PATH_INFO: `/v3/projects/2a805612e76c4f5e9052aa4a8ead24f1` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:30.114303 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1218f-b053-4c6c-9b30-2656578d5c6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a805612e76c4f5e9052aa4a8ead24f1)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:30.116389 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1218f-b053-4c6c-9b30-2656578d5c6f admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:30.179453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:30.181164 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-33d1218f-b053-4c6c-9b30-2656578d5c6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a805612e76c4f5e9052aa4a8ead24f1', 'request_id': 'req-33d1218f-b053-4c6c-9b30-2656578d5c6f'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:30.186184 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc40c51-852e-4696-a2cc-9dc4691b76e0 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:30.189061 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-33d1218f-b053-4c6c-9b30-2656578d5c6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a805612e76c4f5e9052aa4a8ead24f1', 'request_id': 'req-33d1218f-b053-4c6c-9b30-2656578d5c6f'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:30.196236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-33d1218f-b053-4c6c-9b30-2656578d5c6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a805612e76c4f5e9052aa4a8ead24f1', 'request_id': 'req-33d1218f-b053-4c6c-9b30-2656578d5c6f'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:30.215937 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-33d1218f-b053-4c6c-9b30-2656578d5c6f admin admin] The token cache is being invalidate because project 2a805612e76c4f5e9052aa4a8ead24f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:30.216349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-33d1218f-b053-4c6c-9b30-2656578d5c6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:30.217037 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-33d1218f-b053-4c6c-9b30-2656578d5c6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:30.218982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1110/2246] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:29 2022] DELETE /identity/v3/projects/2a805612e76c4f5e9052aa4a8ead24f1 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:30.219802 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1218f-b053-4c6c-9b30-2656578d5c6f admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:30.221764 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-33d1218f-b053-4c6c-9b30-2656578d5c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:30.278716 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9500a4-ae6c-47b0-93ec-3c0dcf9ed933 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e83ea69ded040448fe75caa221ad377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:30.279300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9500a4-ae6c-47b0-93ec-3c0dcf9ed933 service swift] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:30.279685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9500a4-ae6c-47b0-93ec-3c0dcf9ed933 service swift] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:30.279897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9500a4-ae6c-47b0-93ec-3c0dcf9ed933 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:30.282373 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-ef9500a4-ae6c-47b0-93ec-3c0dcf9ed933 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:30.289980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1218f-b053-4c6c-9b30-2656578d5c6f admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:30.293625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3fea4-ba4a-4ae9-afb2-9dc797d1ca73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:30.294469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3fea4-ba4a-4ae9-afb2-9dc797d1ca73 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:30.294730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3fea4-ba4a-4ae9-afb2-9dc797d1ca73 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:30.295021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3fea4-ba4a-4ae9-afb2-9dc797d1ca73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:30.297615 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-91a3fea4-ba4a-4ae9-afb2-9dc797d1ca73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:30.312543 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:30.364231 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9500a4-ae6c-47b0-93ec-3c0dcf9ed933 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:30.366675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9500a4-ae6c-47b0-93ec-3c0dcf9ed933 service swift] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:30.376972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1137/2247] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jun 27 15:49:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 15:49:30.378322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9201b9-f9fa-4b6b-85f4-cca7edf0aa5d None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:30.378755 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9201b9-f9fa-4b6b-85f4-cca7edf0aa5d None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:30.379045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9201b9-f9fa-4b6b-85f4-cca7edf0aa5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:30.380957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3fea4-ba4a-4ae9-afb2-9dc797d1ca73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:30.383006 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3fea4-ba4a-4ae9-afb2-9dc797d1ca73 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:30.394704 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1111/2248] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:30.416839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-1b9201b9-f9fa-4b6b-85f4-cca7edf0aa5d None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:30.420187 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1b9201b9-f9fa-4b6b-85f4-cca7edf0aa5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:30.540596 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1138/2249] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:30.543438 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a6804-ddcd-4fb5-aa9c-6414641cb181 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:30.543931 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a6804-ddcd-4fb5-aa9c-6414641cb181 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:30.544404 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a6804-ddcd-4fb5-aa9c-6414641cb181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:30.603258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-950a6804-ddcd-4fb5-aa9c-6414641cb181 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:30.605254 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-950a6804-ddcd-4fb5-aa9c-6414641cb181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:30.627305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1112/2250] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:30.632316 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9201b9-f9fa-4b6b-85f4-cca7edf0aa5d None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:30.646504 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9201b9-f9fa-4b6b-85f4-cca7edf0aa5d None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:30.666100 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06144634-88af-4149-aa80-362953b86a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:30.667041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06144634-88af-4149-aa80-362953b86a70 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:30.671868 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06144634-88af-4149-aa80-362953b86a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:30.672160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06144634-88af-4149-aa80-362953b86a70 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:30.672957 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06144634-88af-4149-aa80-362953b86a70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:30.673276 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06144634-88af-4149-aa80-362953b86a70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:30.677445 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06144634-88af-4149-aa80-362953b86a70 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:30.712384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1139/2251] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:49:30.717520 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a6804-ddcd-4fb5-aa9c-6414641cb181 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:30.740921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a6804-ddcd-4fb5-aa9c-6414641cb181 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:30.746816 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea3460-2e36-41e7-9e31-be082c963e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:30.747652 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea3460-2e36-41e7-9e31-be082c963e20 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:30.748008 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea3460-2e36-41e7-9e31-be082c963e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:30.748361 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea3460-2e36-41e7-9e31-be082c963e20 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:30.749615 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea3460-2e36-41e7-9e31-be082c963e20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:30.751872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea3460-2e36-41e7-9e31-be082c963e20 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:30.796766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1113/2252] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:30 2022] POST /identity/v3/projects => generated 387 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:30.805769 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06144634-88af-4149-aa80-362953b86a70 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:30.834575 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06144634-88af-4149-aa80-362953b86a70 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:30.906222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:30.942848 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd20b1-da4e-4696-bc1b-091038f75572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:30.943615 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd20b1-da4e-4696-bc1b-091038f75572 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:30.943975 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd20b1-da4e-4696-bc1b-091038f75572 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:30.944329 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd20b1-da4e-4696-bc1b-091038f75572 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:30.945386 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd20b1-da4e-4696-bc1b-091038f75572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:30.948279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd20b1-da4e-4696-bc1b-091038f75572 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:30.970395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1140/2253] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:30 2022] POST /identity/v3/users => generated 294 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:30.976227 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea3460-2e36-41e7-9e31-be082c963e20 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:30.999149 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea3460-2e36-41e7-9e31-be082c963e20 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:31.004072 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1c4d5-e77c-4928-a5c4-61b5fa86b83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:31.005088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1c4d5-e77c-4928-a5c4-61b5fa86b83e admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:31.005088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1c4d5-e77c-4928-a5c4-61b5fa86b83e admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:31.005402 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1c4d5-e77c-4928-a5c4-61b5fa86b83e admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:31.006129 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1c4d5-e77c-4928-a5c4-61b5fa86b83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:31.007741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1c4d5-e77c-4928-a5c4-61b5fa86b83e admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:31.020143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1114/2254] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:31.024269 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd20b1-da4e-4696-bc1b-091038f75572 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:31.034752 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd20b1-da4e-4696-bc1b-091038f75572 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:31.041133 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb58e7-ee1d-4d52-ac3e-7547b12e9baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:31.041896 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb58e7-ee1d-4d52-ac3e-7547b12e9baf admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:31.042250 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb58e7-ee1d-4d52-ac3e-7547b12e9baf admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:31.042569 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb58e7-ee1d-4d52-ac3e-7547b12e9baf admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:31.043307 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb58e7-ee1d-4d52-ac3e-7547b12e9baf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:31.045508 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb58e7-ee1d-4d52-ac3e-7547b12e9baf admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:31.059865 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1141/2255] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:49:31.063713 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1c4d5-e77c-4928-a5c4-61b5fa86b83e admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:31.074986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1c4d5-e77c-4928-a5c4-61b5fa86b83e admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:31.080898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a96295-591e-4014-bd39-b7d4ce9c08f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:31.082142 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a96295-591e-4014-bd39-b7d4ce9c08f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:31.082546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a96295-591e-4014-bd39-b7d4ce9c08f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:31.082903 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a96295-591e-4014-bd39-b7d4ce9c08f0 admin admin] PATH_INFO: `/v3/projects/5d416620e9f942ab9f76e6b117cd8b61/users/6890639a0a2e4ca9b103919fdbbffc5b/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:31.103815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a96295-591e-4014-bd39-b7d4ce9c08f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d416620e9f942ab9f76e6b117cd8b61, user_id=6890639a0a2e4ca9b103919fdbbffc5b, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:31.106534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a96295-591e-4014-bd39-b7d4ce9c08f0 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:31.126889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1115/2256] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:31 2022] PUT /identity/v3/projects/5d416620e9f942ab9f76e6b117cd8b61/users/6890639a0a2e4ca9b103919fdbbffc5b/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:31.128230 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04623b12-4e9f-4fc5-8b74-3cb2e618a945 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:31.128645 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04623b12-4e9f-4fc5-8b74-3cb2e618a945 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:31.128732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4426c-40cb-4482-b61e-440b64d18352 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:31.128817 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04623b12-4e9f-4fc5-8b74-3cb2e618a945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:31.128913 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4426c-40cb-4482-b61e-440b64d18352 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:31.129367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4426c-40cb-4482-b61e-440b64d18352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:31.149550 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-04623b12-4e9f-4fc5-8b74-3cb2e618a945 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:31.150976 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-04623b12-4e9f-4fc5-8b74-3cb2e618a945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:31.153887 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-32d4426c-40cb-4482-b61e-440b64d18352 None None] MFA Rules not processed for user `6655366e7eb84a26aa3f7fbe88af62b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:31.155087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-32d4426c-40cb-4482-b61e-440b64d18352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:31.231030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1142/2257] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:31.235711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1116/2258] 10.222.0.20 () {58 vars in 967 bytes} [Mon Jun 27 15:49:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:31.240047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04623b12-4e9f-4fc5-8b74-3cb2e618a945 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:31.241586 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-04623b12-4e9f-4fc5-8b74-3cb2e618a945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:31.253716 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4426c-40cb-4482-b61e-440b64d18352 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:31.255697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-32d4426c-40cb-4482-b61e-440b64d18352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:31.313302 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04623b12-4e9f-4fc5-8b74-3cb2e618a945 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:31.316565 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:31.316950 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89cdace-3380-4ac4-a490-fb70f6fd5623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:31.317751 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cdace-3380-4ac4-a490-fb70f6fd5623 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:31.317977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cdace-3380-4ac4-a490-fb70f6fd5623 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:31.318337 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cdace-3380-4ac4-a490-fb70f6fd5623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:31.329924 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89cdace-3380-4ac4-a490-fb70f6fd5623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:31.331910 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89cdace-3380-4ac4-a490-fb70f6fd5623 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:31.346312 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4426c-40cb-4482-b61e-440b64d18352 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:31.352915 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725d93f-ad96-4c97-aa8a-35f60d2e3f12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e83ea69ded040448fe75caa221ad377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:31.353582 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725d93f-ad96-4c97-aa8a-35f60d2e3f12 service swift] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:31.354001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725d93f-ad96-4c97-aa8a-35f60d2e3f12 service swift] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:31.354279 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725d93f-ad96-4c97-aa8a-35f60d2e3f12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:31.357708 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1143/2259] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:31.363627 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725d93f-ad96-4c97-aa8a-35f60d2e3f12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:31.366051 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725d93f-ad96-4c97-aa8a-35f60d2e3f12 service swift] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:31.376187 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1117/2260] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jun 27 15:49:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 15:49:31.690138 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d433f-31c0-4e3e-a266-9e4316081567 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:31.691298 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d433f-31c0-4e3e-a266-9e4316081567 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:31.691725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d433f-31c0-4e3e-a266-9e4316081567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:31.724580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-879d433f-31c0-4e3e-a266-9e4316081567 None None] MFA Rules not processed for user `6655366e7eb84a26aa3f7fbe88af62b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:31.725623 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-879d433f-31c0-4e3e-a266-9e4316081567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:31.752192 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1144/2261] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jun 27 15:49:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:31.766193 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725d93f-ad96-4c97-aa8a-35f60d2e3f12 service swift] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:31.783661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725d93f-ad96-4c97-aa8a-35f60d2e3f12 service swift] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:31.791874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5b09e-437c-4820-8b46-b32dce53ec4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e83ea69ded040448fe75caa221ad377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:31.793082 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5b09e-437c-4820-8b46-b32dce53ec4f service swift] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:31.793402 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5b09e-437c-4820-8b46-b32dce53ec4f service swift] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:31.793870 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5b09e-437c-4820-8b46-b32dce53ec4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:31.814696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5b09e-437c-4820-8b46-b32dce53ec4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:31.816280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5b09e-437c-4820-8b46-b32dce53ec4f service swift] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:31.828845 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1118/2262] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jun 27 15:49:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:49:32.541413 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67752ddc-14bb-4569-bea9-ce88ee7e6251 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:32.541926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67752ddc-14bb-4569-bea9-ce88ee7e6251 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:32.542115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67752ddc-14bb-4569-bea9-ce88ee7e6251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:32.565533 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-67752ddc-14bb-4569-bea9-ce88ee7e6251 None None] MFA Rules not processed for user `6655366e7eb84a26aa3f7fbe88af62b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:32.567646 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-67752ddc-14bb-4569-bea9-ce88ee7e6251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:32.577010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1145/2263] 10.222.0.20 () {58 vars in 967 bytes} [Mon Jun 27 15:49:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:32.595347 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5b09e-437c-4820-8b46-b32dce53ec4f service swift] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:32.607526 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5b09e-437c-4820-8b46-b32dce53ec4f service swift] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:32.613067 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b93c0-b011-4dd1-9db0-a816e3006d41 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e83ea69ded040448fe75caa221ad377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:32.614591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b93c0-b011-4dd1-9db0-a816e3006d41 service swift] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:32.615068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b93c0-b011-4dd1-9db0-a816e3006d41 service swift] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:32.615524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b93c0-b011-4dd1-9db0-a816e3006d41 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:32.625777 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b93c0-b011-4dd1-9db0-a816e3006d41 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:32.627675 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b93c0-b011-4dd1-9db0-a816e3006d41 service swift] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:32.637808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1119/2264] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jun 27 15:49:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:49:35.189683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67752ddc-14bb-4569-bea9-ce88ee7e6251 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:35.191269 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-67752ddc-14bb-4569-bea9-ce88ee7e6251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:35.212640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67752ddc-14bb-4569-bea9-ce88ee7e6251 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:35.218387 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eccb0a1-a337-4f1d-890d-c39ce338cebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:35.219558 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccb0a1-a337-4f1d-890d-c39ce338cebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:35.219839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccb0a1-a337-4f1d-890d-c39ce338cebc admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:35.220115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccb0a1-a337-4f1d-890d-c39ce338cebc admin admin] PATH_INFO: `/v3/users/b22e385b694c4a35a7727c4f8d4a080d` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:35.221368 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eccb0a1-a337-4f1d-890d-c39ce338cebc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b22e385b694c4a35a7727c4f8d4a080d)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:35.223890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eccb0a1-a337-4f1d-890d-c39ce338cebc admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:35.273055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:35.293190 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-0eccb0a1-a337-4f1d-890d-c39ce338cebc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b22e385b694c4a35a7727c4f8d4a080d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:35.318255 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-0eccb0a1-a337-4f1d-890d-c39ce338cebc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b22e385b694c4a35a7727c4f8d4a080d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:35.319177 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-0eccb0a1-a337-4f1d-890d-c39ce338cebc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b22e385b694c4a35a7727c4f8d4a080d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:35.327239 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b93c0-b011-4dd1-9db0-a816e3006d41 service swift] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:35.329145 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-861b93c0-b011-4dd1-9db0-a816e3006d41 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:35.332112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-0eccb0a1-a337-4f1d-890d-c39ce338cebc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b22e385b694c4a35a7727c4f8d4a080d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:35.341566 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-0eccb0a1-a337-4f1d-890d-c39ce338cebc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b22e385b694c4a35a7727c4f8d4a080d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:35.348966 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1146/2265] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:35 2022] DELETE /identity/v3/users/b22e385b694c4a35a7727c4f8d4a080d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:35.350980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b93c0-b011-4dd1-9db0-a816e3006d41 service swift] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:35.355723 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43967f-1e5b-4444-8660-9c55db3115f9 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:35.356013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43967f-1e5b-4444-8660-9c55db3115f9 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:35.356225 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f43967f-1e5b-4444-8660-9c55db3115f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:35.382382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-6f43967f-1e5b-4444-8660-9c55db3115f9 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:35.384104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-6f43967f-1e5b-4444-8660-9c55db3115f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:35.433301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c4025-ff39-4be2-b57b-af56a7bb56db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:35.434263 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c4025-ff39-4be2-b57b-af56a7bb56db admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:35.434796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c4025-ff39-4be2-b57b-af56a7bb56db admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:35.435147 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c4025-ff39-4be2-b57b-af56a7bb56db admin admin] PATH_INFO: `/v3/users/364c6c6f607045d988dd55d8f467b027` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:35.436500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c4025-ff39-4be2-b57b-af56a7bb56db admin admin] RBAC: Authorizing `identity:delete_user(user_id=364c6c6f607045d988dd55d8f467b027)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:35.438671 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c4025-ff39-4be2-b57b-af56a7bb56db admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:35.464014 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1147/2266] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:35.479019 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f43967f-1e5b-4444-8660-9c55db3115f9 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:35.481030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-6f43967f-1e5b-4444-8660-9c55db3115f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:35.524898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-262c4025-ff39-4be2-b57b-af56a7bb56db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '364c6c6f607045d988dd55d8f467b027'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:35.544756 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-262c4025-ff39-4be2-b57b-af56a7bb56db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '364c6c6f607045d988dd55d8f467b027'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:35.561920 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-262c4025-ff39-4be2-b57b-af56a7bb56db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '364c6c6f607045d988dd55d8f467b027'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:35.566358 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-262c4025-ff39-4be2-b57b-af56a7bb56db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '364c6c6f607045d988dd55d8f467b027'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:35.568275 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-262c4025-ff39-4be2-b57b-af56a7bb56db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '364c6c6f607045d988dd55d8f467b027'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:35.579936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f43967f-1e5b-4444-8660-9c55db3115f9 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:35.585414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca88d7-da59-47fa-aa7a-5a019e4e0554 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:35.586538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca88d7-da59-47fa-aa7a-5a019e4e0554 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:35.586782 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca88d7-da59-47fa-aa7a-5a019e4e0554 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:35.587071 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca88d7-da59-47fa-aa7a-5a019e4e0554 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:35.588907 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1120/2267] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:35 2022] DELETE /identity/v3/users/364c6c6f607045d988dd55d8f467b027 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:35.589700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2bca88d7-da59-47fa-aa7a-5a019e4e0554 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:35.594266 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeed98-eaa5-4b62-81dd-2ce88bc74c87 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:35.594733 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeed98-eaa5-4b62-81dd-2ce88bc74c87 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:35.595186 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeed98-eaa5-4b62-81dd-2ce88bc74c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:35.630539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-ecdeed98-eaa5-4b62-81dd-2ce88bc74c87 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:35.631806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-ecdeed98-eaa5-4b62-81dd-2ce88bc74c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:35.643496 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca88d7-da59-47fa-aa7a-5a019e4e0554 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:35.648507 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca88d7-da59-47fa-aa7a-5a019e4e0554 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:35.671877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1148/2268] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:35.703821 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:35.759544 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1121/2269] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:35.790324 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca88d7-da59-47fa-aa7a-5a019e4e0554 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:35.793218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2bca88d7-da59-47fa-aa7a-5a019e4e0554 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:35.814267 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca88d7-da59-47fa-aa7a-5a019e4e0554 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:35.935500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c36ab2-bab4-4109-a20d-b7606c517e2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:35.941501 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c36ab2-bab4-4109-a20d-b7606c517e2a service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:35.942340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c36ab2-bab4-4109-a20d-b7606c517e2a service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:35.945709 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c36ab2-bab4-4109-a20d-b7606c517e2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:35.967433 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c36ab2-bab4-4109-a20d-b7606c517e2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:35.970876 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c36ab2-bab4-4109-a20d-b7606c517e2a service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:35.984355 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:36.008467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1149/2270] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:49:36.474294 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdeed98-eaa5-4b62-81dd-2ce88bc74c87 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:36.476958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-ecdeed98-eaa5-4b62-81dd-2ce88bc74c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:36.495366 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdeed98-eaa5-4b62-81dd-2ce88bc74c87 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:36.501276 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040d7c6-71c6-4dee-a0d9-ea8acca4cbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:36.502278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040d7c6-71c6-4dee-a0d9-ea8acca4cbb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:36.502563 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040d7c6-71c6-4dee-a0d9-ea8acca4cbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:36.502842 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040d7c6-71c6-4dee-a0d9-ea8acca4cbb9 admin admin] PATH_INFO: `/v3/projects/c630467b763a410e936008052e5fdb08` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:36.504846 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040d7c6-71c6-4dee-a0d9-ea8acca4cbb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c630467b763a410e936008052e5fdb08)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:36.507600 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040d7c6-71c6-4dee-a0d9-ea8acca4cbb9 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:36.573194 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b040d7c6-71c6-4dee-a0d9-ea8acca4cbb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c630467b763a410e936008052e5fdb08', 'request_id': 'req-b040d7c6-71c6-4dee-a0d9-ea8acca4cbb9'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:36.592305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b040d7c6-71c6-4dee-a0d9-ea8acca4cbb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c630467b763a410e936008052e5fdb08', 'request_id': 'req-b040d7c6-71c6-4dee-a0d9-ea8acca4cbb9'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:36.604299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b040d7c6-71c6-4dee-a0d9-ea8acca4cbb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c630467b763a410e936008052e5fdb08', 'request_id': 'req-b040d7c6-71c6-4dee-a0d9-ea8acca4cbb9'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:36.621375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b040d7c6-71c6-4dee-a0d9-ea8acca4cbb9 admin admin] The token cache is being invalidate because project c630467b763a410e936008052e5fdb08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:36.622481 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b040d7c6-71c6-4dee-a0d9-ea8acca4cbb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:36.623609 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b040d7c6-71c6-4dee-a0d9-ea8acca4cbb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:36.630174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c36ab2-bab4-4109-a20d-b7606c517e2a service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:36.631569 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1122/2271] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:36 2022] DELETE /identity/v3/projects/c630467b763a410e936008052e5fdb08 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:36.643402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c36ab2-bab4-4109-a20d-b7606c517e2a service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:36.745822 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb96ae3-77c1-4da6-8016-0b82804aa345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:36.745822 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb96ae3-77c1-4da6-8016-0b82804aa345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:36.745822 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb96ae3-77c1-4da6-8016-0b82804aa345 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:36.745822 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb96ae3-77c1-4da6-8016-0b82804aa345 admin admin] PATH_INFO: `/v3/users/91875588184340eda0e8ba0c449ee49c` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:36.748375 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb96ae3-77c1-4da6-8016-0b82804aa345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91875588184340eda0e8ba0c449ee49c)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:36.750915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb96ae3-77c1-4da6-8016-0b82804aa345 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:36.777296 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040d7c6-71c6-4dee-a0d9-ea8acca4cbb9 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:36.779808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-b040d7c6-71c6-4dee-a0d9-ea8acca4cbb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:36.807062 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040d7c6-71c6-4dee-a0d9-ea8acca4cbb9 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:36.814489 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5407c27-d50d-49d4-8a07-f363b51f1287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:36.815760 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407c27-d50d-49d4-8a07-f363b51f1287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:36.816114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407c27-d50d-49d4-8a07-f363b51f1287 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:36.816405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407c27-d50d-49d4-8a07-f363b51f1287 admin admin] PATH_INFO: `/v3/projects/009dfcdbdd3840189d5222ff1606f4fa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:36.819770 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5407c27-d50d-49d4-8a07-f363b51f1287 admin admin] RBAC: Authorizing `identity:delete_project(project_id=009dfcdbdd3840189d5222ff1606f4fa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:36.821793 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5407c27-d50d-49d4-8a07-f363b51f1287 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:36.846995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5bb96ae3-77c1-4da6-8016-0b82804aa345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91875588184340eda0e8ba0c449ee49c'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:36.871781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5bb96ae3-77c1-4da6-8016-0b82804aa345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91875588184340eda0e8ba0c449ee49c'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:36.872684 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5bb96ae3-77c1-4da6-8016-0b82804aa345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91875588184340eda0e8ba0c449ee49c'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:36.882860 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5bb96ae3-77c1-4da6-8016-0b82804aa345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91875588184340eda0e8ba0c449ee49c'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:36.897028 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5bb96ae3-77c1-4da6-8016-0b82804aa345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91875588184340eda0e8ba0c449ee49c'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:36.908482 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1150/2272] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:36 2022] DELETE /identity/v3/users/91875588184340eda0e8ba0c449ee49c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:36.932563 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b5407c27-d50d-49d4-8a07-f363b51f1287 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '009dfcdbdd3840189d5222ff1606f4fa', 'request_id': 'req-b5407c27-d50d-49d4-8a07-f363b51f1287'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:36.943757 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b5407c27-d50d-49d4-8a07-f363b51f1287 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '009dfcdbdd3840189d5222ff1606f4fa', 'request_id': 'req-b5407c27-d50d-49d4-8a07-f363b51f1287'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:36.951306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b5407c27-d50d-49d4-8a07-f363b51f1287 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '009dfcdbdd3840189d5222ff1606f4fa', 'request_id': 'req-b5407c27-d50d-49d4-8a07-f363b51f1287'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:36.971423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b5407c27-d50d-49d4-8a07-f363b51f1287 admin admin] The token cache is being invalidate because project 009dfcdbdd3840189d5222ff1606f4fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:36.972205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b5407c27-d50d-49d4-8a07-f363b51f1287 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:36.973868 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b5407c27-d50d-49d4-8a07-f363b51f1287 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:36.976906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1123/2273] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:36 2022] DELETE /identity/v3/projects/009dfcdbdd3840189d5222ff1606f4fa => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:36.983003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d12bb-c36b-4017-9313-af9e3f9106f2 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:36.983342 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d12bb-c36b-4017-9313-af9e3f9106f2 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:36.983672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d12bb-c36b-4017-9313-af9e3f9106f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:37.025275 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-052d12bb-c36b-4017-9313-af9e3f9106f2 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:37.026546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-052d12bb-c36b-4017-9313-af9e3f9106f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:37.150483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1151/2274] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:37.168638 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a73d9-258f-4fee-ae81-a1526fc9a70e None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:37.168928 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a73d9-258f-4fee-ae81-a1526fc9a70e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:37.169261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a73d9-258f-4fee-ae81-a1526fc9a70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:37.199094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-c54a73d9-258f-4fee-ae81-a1526fc9a70e None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:37.200352 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-c54a73d9-258f-4fee-ae81-a1526fc9a70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:37.214012 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1124/2275] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:37.219017 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d12bb-c36b-4017-9313-af9e3f9106f2 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:37.243885 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d12bb-c36b-4017-9313-af9e3f9106f2 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:37.253885 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da535a71-0205-4b27-a7ce-d81acabcbb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:37.256107 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da535a71-0205-4b27-a7ce-d81acabcbb02 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:37.256399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da535a71-0205-4b27-a7ce-d81acabcbb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:37.256683 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da535a71-0205-4b27-a7ce-d81acabcbb02 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:37.257800 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da535a71-0205-4b27-a7ce-d81acabcbb02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:37.258119 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da535a71-0205-4b27-a7ce-d81acabcbb02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:37.260339 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da535a71-0205-4b27-a7ce-d81acabcbb02 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:37.276688 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1152/2276] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:49:37.290049 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a73d9-258f-4fee-ae81-a1526fc9a70e None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:37.313188 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a73d9-258f-4fee-ae81-a1526fc9a70e None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:37.326312 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5eefa-9af2-4172-8558-d063da79fdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:37.326891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5eefa-9af2-4172-8558-d063da79fdb7 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:37.327188 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5eefa-9af2-4172-8558-d063da79fdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:37.327491 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5eefa-9af2-4172-8558-d063da79fdb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:37.328195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5eefa-9af2-4172-8558-d063da79fdb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:37.330904 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5eefa-9af2-4172-8558-d063da79fdb7 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:37.366709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1125/2277] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:37 2022] POST /identity/v3/projects => generated 387 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:37.371133 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da535a71-0205-4b27-a7ce-d81acabcbb02 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:37.384453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da535a71-0205-4b27-a7ce-d81acabcbb02 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:37.389050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:37.505934 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ce0dd-d3e8-4111-bb9c-e6c963463c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:37.507185 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ce0dd-d3e8-4111-bb9c-e6c963463c57 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:37.507185 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ce0dd-d3e8-4111-bb9c-e6c963463c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:37.507368 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ce0dd-d3e8-4111-bb9c-e6c963463c57 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:37.509946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ce0dd-d3e8-4111-bb9c-e6c963463c57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:37.511436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ce0dd-d3e8-4111-bb9c-e6c963463c57 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:37.554627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1153/2278] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:37 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:37.559102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5eefa-9af2-4172-8558-d063da79fdb7 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:37.584549 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5eefa-9af2-4172-8558-d063da79fdb7 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:37.602152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb4a0a-b672-47af-a58a-3d2d5e056bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:37.603002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb4a0a-b672-47af-a58a-3d2d5e056bab admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:37.603363 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb4a0a-b672-47af-a58a-3d2d5e056bab admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:37.603721 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb4a0a-b672-47af-a58a-3d2d5e056bab admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:37.604723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb4a0a-b672-47af-a58a-3d2d5e056bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:37.606596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb4a0a-b672-47af-a58a-3d2d5e056bab admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:37.614856 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:37.621839 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1126/2279] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:37.626379 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ce0dd-d3e8-4111-bb9c-e6c963463c57 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:37.644355 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ce0dd-d3e8-4111-bb9c-e6c963463c57 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:37.651552 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d1ab9-1262-4d05-97f6-ca891cc0cf50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:37.652434 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d1ab9-1262-4d05-97f6-ca891cc0cf50 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:37.652790 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d1ab9-1262-4d05-97f6-ca891cc0cf50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:37.653139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d1ab9-1262-4d05-97f6-ca891cc0cf50 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:37.654107 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d1ab9-1262-4d05-97f6-ca891cc0cf50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:37.656235 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438d1ab9-1262-4d05-97f6-ca891cc0cf50 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:37.673173 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1154/2280] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:37.674647 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb4a0a-b672-47af-a58a-3d2d5e056bab admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:37.685905 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb4a0a-b672-47af-a58a-3d2d5e056bab admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:37.692571 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c44249-56e6-4cf3-8f21-019f458f198e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:37.693484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c44249-56e6-4cf3-8f21-019f458f198e admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:37.693830 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c44249-56e6-4cf3-8f21-019f458f198e admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:37.694112 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c44249-56e6-4cf3-8f21-019f458f198e admin admin] PATH_INFO: `/v3/projects/2030656311a042fab4c543eb825733c4/users/08d64e58c8ce4513a98b1aab4519e4f2/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:37.712212 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c44249-56e6-4cf3-8f21-019f458f198e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2030656311a042fab4c543eb825733c4, user_id=08d64e58c8ce4513a98b1aab4519e4f2, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:37.715044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c44249-56e6-4cf3-8f21-019f458f198e admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:37.740178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1127/2281] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:37 2022] PUT /identity/v3/projects/2030656311a042fab4c543eb825733c4/users/08d64e58c8ce4513a98b1aab4519e4f2/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:37.750973 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71351a59-e9ca-47cb-895e-93bf87e10571 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:37.751353 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71351a59-e9ca-47cb-895e-93bf87e10571 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:37.751714 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71351a59-e9ca-47cb-895e-93bf87e10571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:37.788525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-71351a59-e9ca-47cb-895e-93bf87e10571 None None] MFA Rules not processed for user `08d64e58c8ce4513a98b1aab4519e4f2`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:37.790128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-71351a59-e9ca-47cb-895e-93bf87e10571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:37.879873 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c44249-56e6-4cf3-8f21-019f458f198e admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:37.882058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-57c44249-56e6-4cf3-8f21-019f458f198e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:37.921154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1155/2282] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:37.938563 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71351a59-e9ca-47cb-895e-93bf87e10571 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:37.949960 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-71351a59-e9ca-47cb-895e-93bf87e10571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:38.065833 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c44249-56e6-4cf3-8f21-019f458f198e admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:38.068544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71351a59-e9ca-47cb-895e-93bf87e10571 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:38.074300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b9fc7-7d44-4d3a-bd2e-ad90afe569c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:38.075105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b9fc7-7d44-4d3a-bd2e-ad90afe569c9 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:38.075504 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b9fc7-7d44-4d3a-bd2e-ad90afe569c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:38.075860 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b9fc7-7d44-4d3a-bd2e-ad90afe569c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:38.087098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fedea48-12a9-4c0c-a81b-0c1578065122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:38.087098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fedea48-12a9-4c0c-a81b-0c1578065122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:38.087854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b9fc7-7d44-4d3a-bd2e-ad90afe569c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:38.088426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fedea48-12a9-4c0c-a81b-0c1578065122 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:38.089738 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fedea48-12a9-4c0c-a81b-0c1578065122 admin admin] PATH_INFO: `/v3/projects/2e4f8598bff54195ad246fde2b50fe28` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:38.090170 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b9fc7-7d44-4d3a-bd2e-ad90afe569c9 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:38.095270 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fedea48-12a9-4c0c-a81b-0c1578065122 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e4f8598bff54195ad246fde2b50fe28)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:38.108093 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fedea48-12a9-4c0c-a81b-0c1578065122 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:38.113217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1156/2283] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:38.289233 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2fedea48-12a9-4c0c-a81b-0c1578065122 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e4f8598bff54195ad246fde2b50fe28', 'request_id': 'req-2fedea48-12a9-4c0c-a81b-0c1578065122'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:38.303636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2fedea48-12a9-4c0c-a81b-0c1578065122 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e4f8598bff54195ad246fde2b50fe28', 'request_id': 'req-2fedea48-12a9-4c0c-a81b-0c1578065122'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:38.324609 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2fedea48-12a9-4c0c-a81b-0c1578065122 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e4f8598bff54195ad246fde2b50fe28', 'request_id': 'req-2fedea48-12a9-4c0c-a81b-0c1578065122'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:38.344217 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2fedea48-12a9-4c0c-a81b-0c1578065122 admin admin] The token cache is being invalidate because project 2e4f8598bff54195ad246fde2b50fe28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:38.344560 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2fedea48-12a9-4c0c-a81b-0c1578065122 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:38.345393 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2fedea48-12a9-4c0c-a81b-0c1578065122 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:38.349477 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1128/2284] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:37 2022] DELETE /identity/v3/projects/2e4f8598bff54195ad246fde2b50fe28 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:49:38.353971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bbcfb-42da-4092-9514-891434379623 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:38.354290 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bbcfb-42da-4092-9514-891434379623 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:38.355930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bbcfb-42da-4092-9514-891434379623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:38.389964 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-0e0bbcfb-42da-4092-9514-891434379623 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:38.392947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0e0bbcfb-42da-4092-9514-891434379623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:38.423288 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:38.548339 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1157/2285] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:38.556293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3699d2-f5a7-40cd-a2bd-ae738000505b None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:38.556628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3699d2-f5a7-40cd-a2bd-ae738000505b None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:38.556876 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3699d2-f5a7-40cd-a2bd-ae738000505b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:38.581502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-ff3699d2-f5a7-40cd-a2bd-ae738000505b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:38.583466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-ff3699d2-f5a7-40cd-a2bd-ae738000505b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:38.594718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1129/2286] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:38.599562 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bbcfb-42da-4092-9514-891434379623 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:38.616042 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bbcfb-42da-4092-9514-891434379623 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:38.623535 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f64982-8cc9-41d0-a321-751a5c4f84a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:38.626080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f64982-8cc9-41d0-a321-751a5c4f84a6 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:38.626478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f64982-8cc9-41d0-a321-751a5c4f84a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:38.626846 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f64982-8cc9-41d0-a321-751a5c4f84a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:38.627767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f64982-8cc9-41d0-a321-751a5c4f84a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:38.628172 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f64982-8cc9-41d0-a321-751a5c4f84a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:38.632017 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f64982-8cc9-41d0-a321-751a5c4f84a6 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:38.648639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1158/2287] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:49:38.652779 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3699d2-f5a7-40cd-a2bd-ae738000505b None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:38.667936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3699d2-f5a7-40cd-a2bd-ae738000505b None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:38.682649 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5b490-f1d8-4e79-bbdc-9823da541302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:38.683384 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5b490-f1d8-4e79-bbdc-9823da541302 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:38.683871 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5b490-f1d8-4e79-bbdc-9823da541302 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:38.684278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5b490-f1d8-4e79-bbdc-9823da541302 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:38.685213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5b490-f1d8-4e79-bbdc-9823da541302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:38.689011 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5b490-f1d8-4e79-bbdc-9823da541302 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:38.735856 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1130/2288] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:38 2022] POST /identity/v3/projects => generated 389 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:38.739845 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f64982-8cc9-41d0-a321-751a5c4f84a6 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:38.779395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f64982-8cc9-41d0-a321-751a5c4f84a6 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:38.898810 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7a82e-121d-4566-84ec-39d170481b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:38.899550 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7a82e-121d-4566-84ec-39d170481b05 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:38.899833 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7a82e-121d-4566-84ec-39d170481b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:38.900108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7a82e-121d-4566-84ec-39d170481b05 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:38.901587 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7a82e-121d-4566-84ec-39d170481b05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:38.903651 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7a82e-121d-4566-84ec-39d170481b05 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:38.934134 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1159/2289] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:38 2022] POST /identity/v3/users => generated 295 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:49:38.937843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5b490-f1d8-4e79-bbdc-9823da541302 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:38.951025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5b490-f1d8-4e79-bbdc-9823da541302 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:38.964205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b40753-c7d6-4478-99f7-53658526e381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:38.964895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b40753-c7d6-4478-99f7-53658526e381 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:38.965180 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b40753-c7d6-4478-99f7-53658526e381 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:38.965464 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b40753-c7d6-4478-99f7-53658526e381 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:38.966282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b40753-c7d6-4478-99f7-53658526e381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:38.968395 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b40753-c7d6-4478-99f7-53658526e381 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:38.990353 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1131/2290] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:38.994542 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7a82e-121d-4566-84ec-39d170481b05 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:39.008725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7a82e-121d-4566-84ec-39d170481b05 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:39.019445 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7418f-60d7-4615-b207-92a5a6eb10d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:39.020659 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7418f-60d7-4615-b207-92a5a6eb10d6 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:39.021020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7418f-60d7-4615-b207-92a5a6eb10d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:39.021888 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7418f-60d7-4615-b207-92a5a6eb10d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:39.022676 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7418f-60d7-4615-b207-92a5a6eb10d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:39.024475 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7418f-60d7-4615-b207-92a5a6eb10d6 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:39.039170 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1160/2291] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:39.043484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b40753-c7d6-4478-99f7-53658526e381 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:39.055234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b40753-c7d6-4478-99f7-53658526e381 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:39.060211 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8cdfe-44b9-4a36-b181-cd7683dfbed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:39.061114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8cdfe-44b9-4a36-b181-cd7683dfbed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:39.061409 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8cdfe-44b9-4a36-b181-cd7683dfbed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:39.061729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8cdfe-44b9-4a36-b181-cd7683dfbed8 admin admin] PATH_INFO: `/v3/projects/fba946956b5f419783ba0f244ec6ce09/users/d0b0f5c365db46bfbe54000b9663b2d9/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:39.080742 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8cdfe-44b9-4a36-b181-cd7683dfbed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fba946956b5f419783ba0f244ec6ce09, user_id=d0b0f5c365db46bfbe54000b9663b2d9, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:39.084410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8cdfe-44b9-4a36-b181-cd7683dfbed8 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:39.109755 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1132/2292] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:39 2022] PUT /identity/v3/projects/fba946956b5f419783ba0f244ec6ce09/users/d0b0f5c365db46bfbe54000b9663b2d9/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:39.112103 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0af69-47c0-4ad6-a32f-6c9e3498e09e None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:39.112427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0af69-47c0-4ad6-a32f-6c9e3498e09e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:39.112712 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0af69-47c0-4ad6-a32f-6c9e3498e09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:39.145457 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-e6b0af69-47c0-4ad6-a32f-6c9e3498e09e None None] MFA Rules not processed for user `d0b0f5c365db46bfbe54000b9663b2d9`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:39.146906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e6b0af69-47c0-4ad6-a32f-6c9e3498e09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:39.228409 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1161/2293] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:39.229344 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427cff5-45bc-4b3c-8730-e09f13aaffe8 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:39.229831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427cff5-45bc-4b3c-8730-e09f13aaffe8 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:39.230110 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427cff5-45bc-4b3c-8730-e09f13aaffe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:39.233217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0af69-47c0-4ad6-a32f-6c9e3498e09e None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:39.236272 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:39.247364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:39.263952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-b427cff5-45bc-4b3c-8730-e09f13aaffe8 None None] MFA Rules not processed for user `6890639a0a2e4ca9b103919fdbbffc5b`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:39.265256 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-b427cff5-45bc-4b3c-8730-e09f13aaffe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:39.308906 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0af69-47c0-4ad6-a32f-6c9e3498e09e None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:39.313609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbc95a-6f1f-41e9-b0d4-d0713c853305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:39.314304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbc95a-6f1f-41e9-b0d4-d0713c853305 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:39.314591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbc95a-6f1f-41e9-b0d4-d0713c853305 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:39.314931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbc95a-6f1f-41e9-b0d4-d0713c853305 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:39.315543 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbc95a-6f1f-41e9-b0d4-d0713c853305 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:39.318056 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbc95a-6f1f-41e9-b0d4-d0713c853305 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:39.351325 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1133/2294] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:39.368709 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1162/2295] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:39 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:39.369940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427cff5-45bc-4b3c-8730-e09f13aaffe8 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:39.373368 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbc95a-6f1f-41e9-b0d4-d0713c853305 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:39.376949 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-04dbc95a-6f1f-41e9-b0d4-d0713c853305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:39.386157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427cff5-45bc-4b3c-8730-e09f13aaffe8 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:39.470951 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb36157-e78f-4546-a915-6011f143606d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:39.472102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb36157-e78f-4546-a915-6011f143606d admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:39.472395 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb36157-e78f-4546-a915-6011f143606d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:39.472683 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb36157-e78f-4546-a915-6011f143606d admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:39.475298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb36157-e78f-4546-a915-6011f143606d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:39.478395 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb36157-e78f-4546-a915-6011f143606d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:39.497678 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbc95a-6f1f-41e9-b0d4-d0713c853305 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:39.506353 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065940f-fda8-4930-9d5a-c24e2ee69f90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:39.507090 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065940f-fda8-4930-9d5a-c24e2ee69f90 service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:39.507428 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065940f-fda8-4930-9d5a-c24e2ee69f90 service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:39.507715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065940f-fda8-4930-9d5a-c24e2ee69f90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:39.512539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1134/2296] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:39 2022] POST /identity/v3/users => generated 300 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:49:39.515437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb36157-e78f-4546-a915-6011f143606d admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:39.532292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb36157-e78f-4546-a915-6011f143606d admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:39.540547 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b46d9-a7e2-4c57-b2cb-a2429e1797e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:39.541564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b46d9-a7e2-4c57-b2cb-a2429e1797e5 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:39.541898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b46d9-a7e2-4c57-b2cb-a2429e1797e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:39.542161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b46d9-a7e2-4c57-b2cb-a2429e1797e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:39.543169 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b46d9-a7e2-4c57-b2cb-a2429e1797e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:39.545914 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b46d9-a7e2-4c57-b2cb-a2429e1797e5 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:39.558287 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1135/2297] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:39.562383 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b46d9-a7e2-4c57-b2cb-a2429e1797e5 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:39.574342 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b46d9-a7e2-4c57-b2cb-a2429e1797e5 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:39.580400 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428333e9-18d3-45d8-a1b0-748930003179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:39.581464 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428333e9-18d3-45d8-a1b0-748930003179 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:39.581788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428333e9-18d3-45d8-a1b0-748930003179 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:39.582912 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428333e9-18d3-45d8-a1b0-748930003179 admin admin] PATH_INFO: `/v3/projects/bcba619fdc2f47b3a7c773d268bef479/users/16c38177dd8d47da83f98ee4c3010266/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:39.593052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065940f-fda8-4930-9d5a-c24e2ee69f90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:39.594722 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065940f-fda8-4930-9d5a-c24e2ee69f90 service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:39.602946 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428333e9-18d3-45d8-a1b0-748930003179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcba619fdc2f47b3a7c773d268bef479, user_id=16c38177dd8d47da83f98ee4c3010266, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:39.605366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1163/2298] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:49:39.606201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428333e9-18d3-45d8-a1b0-748930003179 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:39.625277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1136/2299] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:39 2022] PUT /identity/v3/projects/bcba619fdc2f47b3a7c773d268bef479/users/16c38177dd8d47da83f98ee4c3010266/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:39.629350 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065940f-fda8-4930-9d5a-c24e2ee69f90 service nova] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:39.641579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065940f-fda8-4930-9d5a-c24e2ee69f90 service nova] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:39.680572 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:39.708896 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82adee-d981-4b96-b461-8a6a1d7ff09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:39.709586 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82adee-d981-4b96-b461-8a6a1d7ff09e admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:39.710381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82adee-d981-4b96-b461-8a6a1d7ff09e admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:39.710381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82adee-d981-4b96-b461-8a6a1d7ff09e admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:39.711044 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82adee-d981-4b96-b461-8a6a1d7ff09e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:39.712573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82adee-d981-4b96-b461-8a6a1d7ff09e admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:39.726982 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1164/2300] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:49:39.731297 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428333e9-18d3-45d8-a1b0-748930003179 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:39.746473 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428333e9-18d3-45d8-a1b0-748930003179 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:39.751964 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963be57-e8e3-431d-b47d-2e719db2be9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:39.752727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963be57-e8e3-431d-b47d-2e719db2be9a admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:39.753075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963be57-e8e3-431d-b47d-2e719db2be9a admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:39.753411 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963be57-e8e3-431d-b47d-2e719db2be9a admin admin] PATH_INFO: `/v3/projects/bcba619fdc2f47b3a7c773d268bef479/users/16c38177dd8d47da83f98ee4c3010266/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:39.756988 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963be57-e8e3-431d-b47d-2e719db2be9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcba619fdc2f47b3a7c773d268bef479, user_id=16c38177dd8d47da83f98ee4c3010266, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:39.760936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963be57-e8e3-431d-b47d-2e719db2be9a admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:39.776848 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1137/2301] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:39 2022] PUT /identity/v3/projects/bcba619fdc2f47b3a7c773d268bef479/users/16c38177dd8d47da83f98ee4c3010266/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:49:39.788947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92d5d0-af94-4974-9414-b0419954db42 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:39.789437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92d5d0-af94-4974-9414-b0419954db42 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:39.789920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92d5d0-af94-4974-9414-b0419954db42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:39.840044 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-1e92d5d0-af94-4974-9414-b0419954db42 None None] MFA Rules not processed for user `16c38177dd8d47da83f98ee4c3010266`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:39.841431 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1e92d5d0-af94-4974-9414-b0419954db42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:40.012571 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1165/2302] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:40.026274 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963be57-e8e3-431d-b47d-2e719db2be9a admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:40.028349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-6963be57-e8e3-431d-b47d-2e719db2be9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:40.105606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:40.137459 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963be57-e8e3-431d-b47d-2e719db2be9a admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:40.142228 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df443d-5342-44b4-9e82-68e828c3b59b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:40.143178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df443d-5342-44b4-9e82-68e828c3b59b service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:40.143733 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df443d-5342-44b4-9e82-68e828c3b59b service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:40.144116 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df443d-5342-44b4-9e82-68e828c3b59b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:40.159373 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df443d-5342-44b4-9e82-68e828c3b59b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:40.164466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df443d-5342-44b4-9e82-68e828c3b59b service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:40.185454 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1138/2303] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:40.431617 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92d5d0-af94-4974-9414-b0419954db42 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:40.550110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92d5d0-af94-4974-9414-b0419954db42 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:40.555428 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e4200-7320-4c66-872e-b0ef288aebe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:40.556391 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e4200-7320-4c66-872e-b0ef288aebe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:40.556760 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e4200-7320-4c66-872e-b0ef288aebe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:40.557108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e4200-7320-4c66-872e-b0ef288aebe5 admin admin] PATH_INFO: `/v3/users/d0b0f5c365db46bfbe54000b9663b2d9` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:40.558724 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e4200-7320-4c66-872e-b0ef288aebe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0b0f5c365db46bfbe54000b9663b2d9)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:40.561199 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e4200-7320-4c66-872e-b0ef288aebe5 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:40.617287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-eb9e4200-7320-4c66-872e-b0ef288aebe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0b0f5c365db46bfbe54000b9663b2d9'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:40.627468 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:40.633475 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-eb9e4200-7320-4c66-872e-b0ef288aebe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0b0f5c365db46bfbe54000b9663b2d9'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:40.634921 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-eb9e4200-7320-4c66-872e-b0ef288aebe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0b0f5c365db46bfbe54000b9663b2d9'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:40.644237 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-eb9e4200-7320-4c66-872e-b0ef288aebe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0b0f5c365db46bfbe54000b9663b2d9'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:40.653318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-eb9e4200-7320-4c66-872e-b0ef288aebe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0b0f5c365db46bfbe54000b9663b2d9'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:40.660323 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1166/2304] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:40 2022] DELETE /identity/v3/users/d0b0f5c365db46bfbe54000b9663b2d9 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:40.665006 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d629e86-08ed-4355-8862-d3e7aad5e9fc None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:40.665513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d629e86-08ed-4355-8862-d3e7aad5e9fc None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:40.665662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d629e86-08ed-4355-8862-d3e7aad5e9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:40.686520 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-4d629e86-08ed-4355-8862-d3e7aad5e9fc None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:40.687879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4d629e86-08ed-4355-8862-d3e7aad5e9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:40.762031 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1139/2305] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:40.768763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e4200-7320-4c66-872e-b0ef288aebe5 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:40.771328 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-eb9e4200-7320-4c66-872e-b0ef288aebe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:40.861328 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e4200-7320-4c66-872e-b0ef288aebe5 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:40.865896 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af505c6-8fa0-4f13-bddb-ead41e2fe6c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:40.866701 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af505c6-8fa0-4f13-bddb-ead41e2fe6c0 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:40.869361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af505c6-8fa0-4f13-bddb-ead41e2fe6c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:40.869361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af505c6-8fa0-4f13-bddb-ead41e2fe6c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:40.900005 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af505c6-8fa0-4f13-bddb-ead41e2fe6c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:40.901706 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af505c6-8fa0-4f13-bddb-ead41e2fe6c0 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:40.928879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1167/2306] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:41.418128 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d629e86-08ed-4355-8862-d3e7aad5e9fc None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:41.419899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4d629e86-08ed-4355-8862-d3e7aad5e9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:41.444632 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d629e86-08ed-4355-8862-d3e7aad5e9fc None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:41.460475 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7310b-3f35-43f8-9760-da4877aac0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:41.460890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7310b-3f35-43f8-9760-da4877aac0c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:41.461179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7310b-3f35-43f8-9760-da4877aac0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:41.461461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7310b-3f35-43f8-9760-da4877aac0c1 admin admin] PATH_INFO: `/v3/projects/fba946956b5f419783ba0f244ec6ce09` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:41.464825 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7310b-3f35-43f8-9760-da4877aac0c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fba946956b5f419783ba0f244ec6ce09)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:41.464825 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7310b-3f35-43f8-9760-da4877aac0c1 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:41.586323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-aed7310b-3f35-43f8-9760-da4877aac0c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fba946956b5f419783ba0f244ec6ce09', 'request_id': 'req-aed7310b-3f35-43f8-9760-da4877aac0c1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:41.595334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-aed7310b-3f35-43f8-9760-da4877aac0c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fba946956b5f419783ba0f244ec6ce09', 'request_id': 'req-aed7310b-3f35-43f8-9760-da4877aac0c1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:41.615105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-aed7310b-3f35-43f8-9760-da4877aac0c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fba946956b5f419783ba0f244ec6ce09', 'request_id': 'req-aed7310b-3f35-43f8-9760-da4877aac0c1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:41.644040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-aed7310b-3f35-43f8-9760-da4877aac0c1 admin admin] The token cache is being invalidate because project fba946956b5f419783ba0f244ec6ce09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:41.644229 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-aed7310b-3f35-43f8-9760-da4877aac0c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:41.647015 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-aed7310b-3f35-43f8-9760-da4877aac0c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:41.649490 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1140/2307] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:41 2022] DELETE /identity/v3/projects/fba946956b5f419783ba0f244ec6ce09 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:41.654850 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af505c6-8fa0-4f13-bddb-ead41e2fe6c0 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:41.669599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af505c6-8fa0-4f13-bddb-ead41e2fe6c0 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:41.781672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650526c-3057-4a28-8314-6de07389a89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:41.782788 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650526c-3057-4a28-8314-6de07389a89c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:41.783203 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650526c-3057-4a28-8314-6de07389a89c admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:41.783484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650526c-3057-4a28-8314-6de07389a89c admin admin] PATH_INFO: `/v3/users/16c38177dd8d47da83f98ee4c3010266` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:41.784810 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650526c-3057-4a28-8314-6de07389a89c admin admin] RBAC: Authorizing `identity:delete_user(user_id=16c38177dd8d47da83f98ee4c3010266)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:41.787256 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650526c-3057-4a28-8314-6de07389a89c admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:41.872675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-0650526c-3057-4a28-8314-6de07389a89c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16c38177dd8d47da83f98ee4c3010266'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:41.932183 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-0650526c-3057-4a28-8314-6de07389a89c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16c38177dd8d47da83f98ee4c3010266'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:41.933012 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-0650526c-3057-4a28-8314-6de07389a89c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16c38177dd8d47da83f98ee4c3010266'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:41.942343 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-0650526c-3057-4a28-8314-6de07389a89c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16c38177dd8d47da83f98ee4c3010266'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:41.966274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-0650526c-3057-4a28-8314-6de07389a89c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16c38177dd8d47da83f98ee4c3010266'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:41.970042 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:41.992651 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1168/2308] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:49:41 2022] DELETE /identity/v3/users/16c38177dd8d47da83f98ee4c3010266 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:49:42.519028 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7310b-3f35-43f8-9760-da4877aac0c1 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:42.522130 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-aed7310b-3f35-43f8-9760-da4877aac0c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:42.557809 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7310b-3f35-43f8-9760-da4877aac0c1 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:42.649109 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a5149-2a0b-4f3b-a77f-0a2136211827 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:42.649400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a5149-2a0b-4f3b-a77f-0a2136211827 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:42.649717 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a5149-2a0b-4f3b-a77f-0a2136211827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:42.671820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-496a5149-2a0b-4f3b-a77f-0a2136211827 None None] MFA Rules not processed for user `6890639a0a2e4ca9b103919fdbbffc5b`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:42.673015 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-496a5149-2a0b-4f3b-a77f-0a2136211827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:42.674833 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b661da-78b3-48cd-b9b3-191233d80847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:42.675840 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b661da-78b3-48cd-b9b3-191233d80847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:42.676168 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b661da-78b3-48cd-b9b3-191233d80847 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:42.676659 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b661da-78b3-48cd-b9b3-191233d80847 admin admin] PATH_INFO: `/v3/projects/bcba619fdc2f47b3a7c773d268bef479` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:42.678612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b661da-78b3-48cd-b9b3-191233d80847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcba619fdc2f47b3a7c773d268bef479)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:42.686083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b661da-78b3-48cd-b9b3-191233d80847 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:42.687921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:42.753192 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-09b661da-78b3-48cd-b9b3-191233d80847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcba619fdc2f47b3a7c773d268bef479', 'request_id': 'req-09b661da-78b3-48cd-b9b3-191233d80847'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:42.756873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1169/2309] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:42.762477 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-09b661da-78b3-48cd-b9b3-191233d80847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcba619fdc2f47b3a7c773d268bef479', 'request_id': 'req-09b661da-78b3-48cd-b9b3-191233d80847'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:42.771656 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-09b661da-78b3-48cd-b9b3-191233d80847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcba619fdc2f47b3a7c773d268bef479', 'request_id': 'req-09b661da-78b3-48cd-b9b3-191233d80847'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:42.794624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-09b661da-78b3-48cd-b9b3-191233d80847 admin admin] The token cache is being invalidate because project bcba619fdc2f47b3a7c773d268bef479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:49:42.795199 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-09b661da-78b3-48cd-b9b3-191233d80847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:42.796571 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-09b661da-78b3-48cd-b9b3-191233d80847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:49:42.799330 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1141/2310] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:49:42 2022] DELETE /identity/v3/projects/bcba619fdc2f47b3a7c773d268bef479 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:42.813890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992836d-1893-4725-b33e-2c05ace652e3 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:42.814817 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992836d-1893-4725-b33e-2c05ace652e3 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:42.815269 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992836d-1893-4725-b33e-2c05ace652e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:42.857549 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-1992836d-1893-4725-b33e-2c05ace652e3 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:42.859573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1992836d-1893-4725-b33e-2c05ace652e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:42.954076 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1170/2311] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:49:42.959599 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d26a25-0ff4-4e0f-862c-15c23a76f627 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:42.959891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d26a25-0ff4-4e0f-862c-15c23a76f627 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:42.960212 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d26a25-0ff4-4e0f-862c-15c23a76f627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:42.981405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-b3d26a25-0ff4-4e0f-862c-15c23a76f627 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:42.991929 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-b3d26a25-0ff4-4e0f-862c-15c23a76f627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:43.003291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1142/2312] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:43.008022 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992836d-1893-4725-b33e-2c05ace652e3 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:43.019365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992836d-1893-4725-b33e-2c05ace652e3 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:43.024872 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599d13e-cc00-4e2d-a712-9b9ddf6e8030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:43.026125 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599d13e-cc00-4e2d-a712-9b9ddf6e8030 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:43.026125 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599d13e-cc00-4e2d-a712-9b9ddf6e8030 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:43.026413 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599d13e-cc00-4e2d-a712-9b9ddf6e8030 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:43.027073 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599d13e-cc00-4e2d-a712-9b9ddf6e8030 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:49:43.027573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599d13e-cc00-4e2d-a712-9b9ddf6e8030 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:43.029326 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599d13e-cc00-4e2d-a712-9b9ddf6e8030 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:43.049174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1171/2313] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:49:43.053272 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d26a25-0ff4-4e0f-862c-15c23a76f627 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:43.073582 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d26a25-0ff4-4e0f-862c-15c23a76f627 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:43.081047 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b889e4f-a568-42db-8b73-0633b1f48554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:43.083447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b889e4f-a568-42db-8b73-0633b1f48554 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:43.084003 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b889e4f-a568-42db-8b73-0633b1f48554 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:43.084831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b889e4f-a568-42db-8b73-0633b1f48554 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:43.085903 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b889e4f-a568-42db-8b73-0633b1f48554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:43.088466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b889e4f-a568-42db-8b73-0633b1f48554 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:43.119226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1143/2314] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:49:43 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:49:43.123718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599d13e-cc00-4e2d-a712-9b9ddf6e8030 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:43.137977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599d13e-cc00-4e2d-a712-9b9ddf6e8030 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:43.219505 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c69e3-c123-4410-861a-fa751777bda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:43.220402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c69e3-c123-4410-861a-fa751777bda3 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:43.220768 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c69e3-c123-4410-861a-fa751777bda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:43.221176 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c69e3-c123-4410-861a-fa751777bda3 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:43.222021 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c69e3-c123-4410-861a-fa751777bda3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:43.224932 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c69e3-c123-4410-861a-fa751777bda3 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:43.252749 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1172/2315] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:49:43 2022] POST /identity/v3/users => generated 288 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:49:43.258210 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b889e4f-a568-42db-8b73-0633b1f48554 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:43.271430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b889e4f-a568-42db-8b73-0633b1f48554 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:43.276908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25088e89-79e1-4ca0-baa7-664bc3aa1b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:43.277788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25088e89-79e1-4ca0-baa7-664bc3aa1b51 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:43.278112 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25088e89-79e1-4ca0-baa7-664bc3aa1b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:43.278443 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25088e89-79e1-4ca0-baa7-664bc3aa1b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:43.279327 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25088e89-79e1-4ca0-baa7-664bc3aa1b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:43.281507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25088e89-79e1-4ca0-baa7-664bc3aa1b51 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:43.298733 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1144/2316] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:49:43.304451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c69e3-c123-4410-861a-fa751777bda3 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:43.315193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c69e3-c123-4410-861a-fa751777bda3 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:43.321289 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f698fe-bdfa-480f-b060-92b31bc4a45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:43.322171 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f698fe-bdfa-480f-b060-92b31bc4a45a admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:43.322599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f698fe-bdfa-480f-b060-92b31bc4a45a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:43.322849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f698fe-bdfa-480f-b060-92b31bc4a45a admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:43.323794 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f698fe-bdfa-480f-b060-92b31bc4a45a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:43.326529 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f698fe-bdfa-480f-b060-92b31bc4a45a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:43.343439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1173/2317] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:49:43.345176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25088e89-79e1-4ca0-baa7-664bc3aa1b51 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:43.356207 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25088e89-79e1-4ca0-baa7-664bc3aa1b51 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:43.361618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76bcd0d-3ead-4fb5-8e26-fba769065c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:43.364133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bcd0d-3ead-4fb5-8e26-fba769065c71 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:43.364133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bcd0d-3ead-4fb5-8e26-fba769065c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:43.364405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bcd0d-3ead-4fb5-8e26-fba769065c71 admin admin] PATH_INFO: `/v3/projects/7c41ab6121304ec491345e780431535a/users/8fc98b9faa9547109246ee8fb7854795/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:43.384123 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76bcd0d-3ead-4fb5-8e26-fba769065c71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c41ab6121304ec491345e780431535a, user_id=8fc98b9faa9547109246ee8fb7854795, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:43.387238 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76bcd0d-3ead-4fb5-8e26-fba769065c71 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:43.426418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1145/2318] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:49:43 2022] PUT /identity/v3/projects/7c41ab6121304ec491345e780431535a/users/8fc98b9faa9547109246ee8fb7854795/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:49:43.428942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebb588-4b09-439e-80b0-19a95f398c06 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:43.429386 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebb588-4b09-439e-80b0-19a95f398c06 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:43.429581 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebb588-4b09-439e-80b0-19a95f398c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:43.451297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:43.469616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-1eebb588-4b09-439e-80b0-19a95f398c06 None None] MFA Rules not processed for user `8fc98b9faa9547109246ee8fb7854795`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:49:43.471053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1eebb588-4b09-439e-80b0-19a95f398c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:43.553399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1174/2319] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:49:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:49:43.565452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76bcd0d-3ead-4fb5-8e26-fba769065c71 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:49:43.567696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d76bcd0d-3ead-4fb5-8e26-fba769065c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:49:43.650872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76bcd0d-3ead-4fb5-8e26-fba769065c71 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:49:43.657227 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639aa55a-9563-4f6c-8dd6-bf10e4053cfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:49:43.657939 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639aa55a-9563-4f6c-8dd6-bf10e4053cfa service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:49:43.658224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639aa55a-9563-4f6c-8dd6-bf10e4053cfa service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:49:43.658499 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639aa55a-9563-4f6c-8dd6-bf10e4053cfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:49:43.670326 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639aa55a-9563-4f6c-8dd6-bf10e4053cfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:49:43.672319 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639aa55a-9563-4f6c-8dd6-bf10e4053cfa service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:49:43.684096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1146/2320] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:49:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:49:53.549649 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=28 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:49:53.678710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:50:04.577136 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebb588-4b09-439e-80b0-19a95f398c06 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:04.578628 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1eebb588-4b09-439e-80b0-19a95f398c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:04.688306 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebb588-4b09-439e-80b0-19a95f398c06 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:04.694491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e1caa-543c-449a-8ffd-748cd67c58b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:04.695524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e1caa-543c-449a-8ffd-748cd67c58b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:04.695896 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e1caa-543c-449a-8ffd-748cd67c58b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:04.696207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e1caa-543c-449a-8ffd-748cd67c58b1 admin admin] PATH_INFO: `/v3/users/08d64e58c8ce4513a98b1aab4519e4f2` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:04.697781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e1caa-543c-449a-8ffd-748cd67c58b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08d64e58c8ce4513a98b1aab4519e4f2)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:04.700117 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e1caa-543c-449a-8ffd-748cd67c58b1 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:04.814400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-770e1caa-543c-449a-8ffd-748cd67c58b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08d64e58c8ce4513a98b1aab4519e4f2'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:04.852725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-770e1caa-543c-449a-8ffd-748cd67c58b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08d64e58c8ce4513a98b1aab4519e4f2'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:04.854044 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-770e1caa-543c-449a-8ffd-748cd67c58b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08d64e58c8ce4513a98b1aab4519e4f2'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:04.867458 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-770e1caa-543c-449a-8ffd-748cd67c58b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08d64e58c8ce4513a98b1aab4519e4f2'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:04.885586 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-770e1caa-543c-449a-8ffd-748cd67c58b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08d64e58c8ce4513a98b1aab4519e4f2'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:04.900192 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1175/2321] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:50:04 2022] DELETE /identity/v3/users/08d64e58c8ce4513a98b1aab4519e4f2 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:50:04.905159 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef30229-551a-4f49-9906-a988f405a2dc None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:04.906478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef30229-551a-4f49-9906-a988f405a2dc None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:04.906988 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef30229-551a-4f49-9906-a988f405a2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:04.935205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-cef30229-551a-4f49-9906-a988f405a2dc None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:50:04.936576 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-cef30229-551a-4f49-9906-a988f405a2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:05.020889 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1147/2322] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:50:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:50:05.031025 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e1caa-543c-449a-8ffd-748cd67c58b1 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:05.032604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-770e1caa-543c-449a-8ffd-748cd67c58b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:05.106666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e1caa-543c-449a-8ffd-748cd67c58b1 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:05.115103 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d2915-fe39-4166-bb96-d7e589a9505b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:05.116093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d2915-fe39-4166-bb96-d7e589a9505b service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:05.116385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d2915-fe39-4166-bb96-d7e589a9505b service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:05.116667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d2915-fe39-4166-bb96-d7e589a9505b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:05.140317 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d2915-fe39-4166-bb96-d7e589a9505b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:05.142337 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d2915-fe39-4166-bb96-d7e589a9505b service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:05.153996 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1176/2323] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:50:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:50:05.453908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef30229-551a-4f49-9906-a988f405a2dc None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:05.455512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-cef30229-551a-4f49-9906-a988f405a2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:05.475397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef30229-551a-4f49-9906-a988f405a2dc None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:05.481992 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01028a8f-db17-4e5a-adde-5fb81b7b1f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:05.482883 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01028a8f-db17-4e5a-adde-5fb81b7b1f1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:05.483169 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01028a8f-db17-4e5a-adde-5fb81b7b1f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:05.483445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01028a8f-db17-4e5a-adde-5fb81b7b1f1d admin admin] PATH_INFO: `/v3/projects/2030656311a042fab4c543eb825733c4` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:05.484703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01028a8f-db17-4e5a-adde-5fb81b7b1f1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2030656311a042fab4c543eb825733c4)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:05.486929 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01028a8f-db17-4e5a-adde-5fb81b7b1f1d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:05.539470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-01028a8f-db17-4e5a-adde-5fb81b7b1f1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2030656311a042fab4c543eb825733c4', 'request_id': 'req-01028a8f-db17-4e5a-adde-5fb81b7b1f1d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:05.545849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-01028a8f-db17-4e5a-adde-5fb81b7b1f1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2030656311a042fab4c543eb825733c4', 'request_id': 'req-01028a8f-db17-4e5a-adde-5fb81b7b1f1d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:05.553045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-01028a8f-db17-4e5a-adde-5fb81b7b1f1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2030656311a042fab4c543eb825733c4', 'request_id': 'req-01028a8f-db17-4e5a-adde-5fb81b7b1f1d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:05.565073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-01028a8f-db17-4e5a-adde-5fb81b7b1f1d admin admin] The token cache is being invalidate because project 2030656311a042fab4c543eb825733c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:50:05.565435 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-01028a8f-db17-4e5a-adde-5fb81b7b1f1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:05.566355 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-01028a8f-db17-4e5a-adde-5fb81b7b1f1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:05.567940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1148/2324] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:50:05 2022] DELETE /identity/v3/projects/2030656311a042fab4c543eb825733c4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:50:05.574448 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d4793-6845-4e62-a747-b2f2aace5554 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:05.574742 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d4793-6845-4e62-a747-b2f2aace5554 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:05.575023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d4793-6845-4e62-a747-b2f2aace5554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:05.593470 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-001d4793-6845-4e62-a747-b2f2aace5554 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:50:05.594757 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-001d4793-6845-4e62-a747-b2f2aace5554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:05.612833 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:50:05.661802 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1177/2325] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:50:05.669660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5203e0a-80da-4e93-8dfd-3520ab6d8e77 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:05.669979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5203e0a-80da-4e93-8dfd-3520ab6d8e77 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:05.670262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5203e0a-80da-4e93-8dfd-3520ab6d8e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:05.690136 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-f5203e0a-80da-4e93-8dfd-3520ab6d8e77 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:50:05.691937 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f5203e0a-80da-4e93-8dfd-3520ab6d8e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:05.700020 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1149/2326] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:50:05.704279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d4793-6845-4e62-a747-b2f2aace5554 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:05.715501 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d4793-6845-4e62-a747-b2f2aace5554 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:05.720179 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e26635-1462-4eb2-b340-4dce5c6142da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:05.720846 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e26635-1462-4eb2-b340-4dce5c6142da admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:05.721131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e26635-1462-4eb2-b340-4dce5c6142da admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:05.721413 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e26635-1462-4eb2-b340-4dce5c6142da admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:05.722378 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e26635-1462-4eb2-b340-4dce5c6142da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:50:05.722643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e26635-1462-4eb2-b340-4dce5c6142da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:05.724596 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e26635-1462-4eb2-b340-4dce5c6142da admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:05.742740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1178/2327] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:50:05.746938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5203e0a-80da-4e93-8dfd-3520ab6d8e77 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:05.757234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5203e0a-80da-4e93-8dfd-3520ab6d8e77 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:05.762112 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5943482-f59c-4a54-8900-8e3b738cd2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:05.762775 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5943482-f59c-4a54-8900-8e3b738cd2d3 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:05.763054 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5943482-f59c-4a54-8900-8e3b738cd2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:05.763334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5943482-f59c-4a54-8900-8e3b738cd2d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:05.764018 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5943482-f59c-4a54-8900-8e3b738cd2d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:05.765956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5943482-f59c-4a54-8900-8e3b738cd2d3 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:05.793678 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1150/2328] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:50:05 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:50:05.801672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e26635-1462-4eb2-b340-4dce5c6142da admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:05.819257 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e26635-1462-4eb2-b340-4dce5c6142da admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:05.892681 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f29ec-625f-45bd-ae24-1e6162feaaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:05.894402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f29ec-625f-45bd-ae24-1e6162feaaed admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:05.894886 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f29ec-625f-45bd-ae24-1e6162feaaed admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:05.895485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f29ec-625f-45bd-ae24-1e6162feaaed admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:05.895810 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f29ec-625f-45bd-ae24-1e6162feaaed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:05.897763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f29ec-625f-45bd-ae24-1e6162feaaed admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:05.923002 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1179/2329] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:50:05 2022] POST /identity/v3/users => generated 288 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:50:05.928938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5943482-f59c-4a54-8900-8e3b738cd2d3 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:05.939252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5943482-f59c-4a54-8900-8e3b738cd2d3 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:05.945417 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaef397-da91-4315-8ac8-82c3f7c0f542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:05.946166 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaef397-da91-4315-8ac8-82c3f7c0f542 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:05.946450 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaef397-da91-4315-8ac8-82c3f7c0f542 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:05.946730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaef397-da91-4315-8ac8-82c3f7c0f542 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:05.947476 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaef397-da91-4315-8ac8-82c3f7c0f542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:05.949094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaef397-da91-4315-8ac8-82c3f7c0f542 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:05.961224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1151/2330] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:50:05.965256 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f29ec-625f-45bd-ae24-1e6162feaaed admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:05.976920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f29ec-625f-45bd-ae24-1e6162feaaed admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:05.981850 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f830adb-c255-4c98-8ba7-6d7ff941c16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:05.982519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f830adb-c255-4c98-8ba7-6d7ff941c16a admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:05.982805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f830adb-c255-4c98-8ba7-6d7ff941c16a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:05.983079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f830adb-c255-4c98-8ba7-6d7ff941c16a admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:05.983816 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f830adb-c255-4c98-8ba7-6d7ff941c16a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:05.985404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f830adb-c255-4c98-8ba7-6d7ff941c16a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:06.008928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1180/2331] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:50:06.013108 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaef397-da91-4315-8ac8-82c3f7c0f542 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:06.024547 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaef397-da91-4315-8ac8-82c3f7c0f542 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:06.030002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba8dd2-10d4-4053-8ed0-0a34ce39e50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:06.030912 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba8dd2-10d4-4053-8ed0-0a34ce39e50e admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:06.031227 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba8dd2-10d4-4053-8ed0-0a34ce39e50e admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:06.031561 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba8dd2-10d4-4053-8ed0-0a34ce39e50e admin admin] PATH_INFO: `/v3/projects/ec9f767e3b4d4fb78f86a611ed9f47c5/users/25024a505319480ab00f4c7e58de600c/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:06.051935 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba8dd2-10d4-4053-8ed0-0a34ce39e50e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec9f767e3b4d4fb78f86a611ed9f47c5, user_id=25024a505319480ab00f4c7e58de600c, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:06.055115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba8dd2-10d4-4053-8ed0-0a34ce39e50e admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:06.071613 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1152/2332] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:50:06 2022] PUT /identity/v3/projects/ec9f767e3b4d4fb78f86a611ed9f47c5/users/25024a505319480ab00f4c7e58de600c/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:50:06.076864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b004491-51b5-42d1-86a9-b9e37bf05273 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:06.077159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b004491-51b5-42d1-86a9-b9e37bf05273 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:06.077466 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b004491-51b5-42d1-86a9-b9e37bf05273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:06.096297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-0b004491-51b5-42d1-86a9-b9e37bf05273 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:50:06.098166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0b004491-51b5-42d1-86a9-b9e37bf05273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:06.170900 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1181/2333] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:50:06.181940 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba8dd2-10d4-4053-8ed0-0a34ce39e50e admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:06.184029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-34ba8dd2-10d4-4053-8ed0-0a34ce39e50e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:06.210447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:50:06.250598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba8dd2-10d4-4053-8ed0-0a34ce39e50e admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:06.254770 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72400a-72c9-4979-a764-9decee868691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:06.255412 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72400a-72c9-4979-a764-9decee868691 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:06.255694 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72400a-72c9-4979-a764-9decee868691 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:06.255966 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72400a-72c9-4979-a764-9decee868691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:06.265277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da72400a-72c9-4979-a764-9decee868691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:06.266746 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da72400a-72c9-4979-a764-9decee868691 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:06.277998 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1153/2334] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:50:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:50:11.355282 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96bf97-dbe4-4164-828a-129f3d4390ed None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:11.355282 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96bf97-dbe4-4164-828a-129f3d4390ed None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:11.355282 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96bf97-dbe4-4164-828a-129f3d4390ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:11.432274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-7d96bf97-dbe4-4164-828a-129f3d4390ed None None] MFA Rules not processed for user `25024a505319480ab00f4c7e58de600c`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:50:11.434216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-7d96bf97-dbe4-4164-828a-129f3d4390ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:11.460710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:50:11.546137 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1182/2335] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:50:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:50:11.550835 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72400a-72c9-4979-a764-9decee868691 service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:11.562635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72400a-72c9-4979-a764-9decee868691 service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:11.574620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f69c7-772f-4290-b2d7-e05592bd830c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:11.575572 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f69c7-772f-4290-b2d7-e05592bd830c admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:11.576054 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f69c7-772f-4290-b2d7-e05592bd830c admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:11.576795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f69c7-772f-4290-b2d7-e05592bd830c admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:11.577775 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f69c7-772f-4290-b2d7-e05592bd830c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:11.580267 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f69c7-772f-4290-b2d7-e05592bd830c admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:11.614348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1154/2336] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:50:11 2022] POST /identity/v3/projects => generated 379 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:50:11.618935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96bf97-dbe4-4164-828a-129f3d4390ed None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:11.631458 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96bf97-dbe4-4164-828a-129f3d4390ed None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:11.701062 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447846f-f7f8-4fc5-8d75-30504d67a831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:11.701926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447846f-f7f8-4fc5-8d75-30504d67a831 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:11.702212 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447846f-f7f8-4fc5-8d75-30504d67a831 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:11.702486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447846f-f7f8-4fc5-8d75-30504d67a831 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:11.703191 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447846f-f7f8-4fc5-8d75-30504d67a831 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:11.706566 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447846f-f7f8-4fc5-8d75-30504d67a831 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:11.736090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f69c7-772f-4290-b2d7-e05592bd830c admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:11.736763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1183/2337] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:50:11 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:50:11.741043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-542f69c7-772f-4290-b2d7-e05592bd830c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:11.741319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447846f-f7f8-4fc5-8d75-30504d67a831 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:11.762116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447846f-f7f8-4fc5-8d75-30504d67a831 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:11.768441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956e4ae-4735-4209-8ec7-bcea9a7d32e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:11.769288 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956e4ae-4735-4209-8ec7-bcea9a7d32e7 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:11.769949 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956e4ae-4735-4209-8ec7-bcea9a7d32e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:11.770246 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956e4ae-4735-4209-8ec7-bcea9a7d32e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:11.772547 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956e4ae-4735-4209-8ec7-bcea9a7d32e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:11.774351 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956e4ae-4735-4209-8ec7-bcea9a7d32e7 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:11.776431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f69c7-772f-4290-b2d7-e05592bd830c admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:11.782662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b368a2-ab22-4111-99dd-6545434b281d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:11.783786 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b368a2-ab22-4111-99dd-6545434b281d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:11.784953 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b368a2-ab22-4111-99dd-6545434b281d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:11.785452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b368a2-ab22-4111-99dd-6545434b281d admin admin] PATH_INFO: `/v3/users/eb2f0ac0e17448f58265379e0ea53b19` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:11.786845 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b368a2-ab22-4111-99dd-6545434b281d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb2f0ac0e17448f58265379e0ea53b19)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:11.791299 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1184/2338] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:50:11.791488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b368a2-ab22-4111-99dd-6545434b281d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:11.792690 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956e4ae-4735-4209-8ec7-bcea9a7d32e7 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:11.811447 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956e4ae-4735-4209-8ec7-bcea9a7d32e7 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:11.833578 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dab695-515d-46c3-bac4-1f1a0f26c806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:11.838442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dab695-515d-46c3-bac4-1f1a0f26c806 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:11.838778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dab695-515d-46c3-bac4-1f1a0f26c806 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:11.839546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dab695-515d-46c3-bac4-1f1a0f26c806 admin admin] PATH_INFO: `/v3/projects/bf5fba05fb7a409c913bab265a982067/users/bf539804a8cf46ada01d8dea4fb1c09f/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:11.869558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-98b368a2-ab22-4111-99dd-6545434b281d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb2f0ac0e17448f58265379e0ea53b19'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:11.870675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dab695-515d-46c3-bac4-1f1a0f26c806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf5fba05fb7a409c913bab265a982067, user_id=bf539804a8cf46ada01d8dea4fb1c09f, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:11.873794 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dab695-515d-46c3-bac4-1f1a0f26c806 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:11.898228 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-98b368a2-ab22-4111-99dd-6545434b281d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb2f0ac0e17448f58265379e0ea53b19'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:11.915967 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-98b368a2-ab22-4111-99dd-6545434b281d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb2f0ac0e17448f58265379e0ea53b19'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:11.917158 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-98b368a2-ab22-4111-99dd-6545434b281d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb2f0ac0e17448f58265379e0ea53b19'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:11.918133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-98b368a2-ab22-4111-99dd-6545434b281d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb2f0ac0e17448f58265379e0ea53b19'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:11.922314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1185/2339] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:50:11 2022] PUT /identity/v3/projects/bf5fba05fb7a409c913bab265a982067/users/bf539804a8cf46ada01d8dea4fb1c09f/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:50:11.927343 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dab695-515d-46c3-bac4-1f1a0f26c806 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:11.929109 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f8dab695-515d-46c3-bac4-1f1a0f26c806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:11.937683 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1155/2340] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:50:11 2022] DELETE /identity/v3/users/eb2f0ac0e17448f58265379e0ea53b19 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:50:11.939933 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdfbad-07bc-4cfd-9b4c-e1de43550334 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:11.941334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdfbad-07bc-4cfd-9b4c-e1de43550334 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:11.941709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdfbad-07bc-4cfd-9b4c-e1de43550334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:11.978871 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-91fdfbad-07bc-4cfd-9b4c-e1de43550334 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:50:11.979535 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-91fdfbad-07bc-4cfd-9b4c-e1de43550334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:12.058544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:50:12.081373 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:50:12.117483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1156/2341] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:50:12.121419 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dab695-515d-46c3-bac4-1f1a0f26c806 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:12.124936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fdfbad-07bc-4cfd-9b4c-e1de43550334 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:12.127423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-91fdfbad-07bc-4cfd-9b4c-e1de43550334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:12.128876 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c856c9-e732-4619-8819-4258c77dc7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:12.129822 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c856c9-e732-4619-8819-4258c77dc7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:12.130180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c856c9-e732-4619-8819-4258c77dc7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:12.130531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c856c9-e732-4619-8819-4258c77dc7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:12.133057 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c856c9-e732-4619-8819-4258c77dc7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:12.142629 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c856c9-e732-4619-8819-4258c77dc7d9 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:12.166598 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1186/2342] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:50:12.170358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c856c9-e732-4619-8819-4258c77dc7d9 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:12.187725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c856c9-e732-4619-8819-4258c77dc7d9 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:12.200907 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41721702-a28b-40cb-8efd-4643cbebed3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:12.202168 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41721702-a28b-40cb-8efd-4643cbebed3f admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:12.202460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41721702-a28b-40cb-8efd-4643cbebed3f admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:12.202746 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41721702-a28b-40cb-8efd-4643cbebed3f admin admin] PATH_INFO: `/v3/projects/bf5fba05fb7a409c913bab265a982067/users/bf539804a8cf46ada01d8dea4fb1c09f/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:12.204833 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41721702-a28b-40cb-8efd-4643cbebed3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf5fba05fb7a409c913bab265a982067, user_id=bf539804a8cf46ada01d8dea4fb1c09f, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:12.208310 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41721702-a28b-40cb-8efd-4643cbebed3f admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:12.224300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1187/2343] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:50:12 2022] PUT /identity/v3/projects/bf5fba05fb7a409c913bab265a982067/users/bf539804a8cf46ada01d8dea4fb1c09f/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:50:12.265810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fdfbad-07bc-4cfd-9b4c-e1de43550334 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:12.336906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6bcab-83d4-45e6-873f-0bba97561b6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:12.338991 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6bcab-83d4-45e6-873f-0bba97561b6d service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:12.339845 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6bcab-83d4-45e6-873f-0bba97561b6d service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:12.340707 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6bcab-83d4-45e6-873f-0bba97561b6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:12.429888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6bcab-83d4-45e6-873f-0bba97561b6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:12.432098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6bcab-83d4-45e6-873f-0bba97561b6d service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:12.446110 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1157/2344] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:50:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:50:12.907941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41721702-a28b-40cb-8efd-4643cbebed3f admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:12.909819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-41721702-a28b-40cb-8efd-4643cbebed3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:12.928965 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41721702-a28b-40cb-8efd-4643cbebed3f admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:12.936967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8248fe-ac8b-43da-833a-8ea7554797b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:12.936967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8248fe-ac8b-43da-833a-8ea7554797b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:12.936967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8248fe-ac8b-43da-833a-8ea7554797b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:12.937359 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8248fe-ac8b-43da-833a-8ea7554797b3 admin admin] PATH_INFO: `/v3/projects/3c6536154b424b40abb6a19fef51867c` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:12.938376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8248fe-ac8b-43da-833a-8ea7554797b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c6536154b424b40abb6a19fef51867c)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:12.941583 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8248fe-ac8b-43da-833a-8ea7554797b3 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:13.013036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ae8248fe-ac8b-43da-833a-8ea7554797b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c6536154b424b40abb6a19fef51867c', 'request_id': 'req-ae8248fe-ac8b-43da-833a-8ea7554797b3'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:13.021200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ae8248fe-ac8b-43da-833a-8ea7554797b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c6536154b424b40abb6a19fef51867c', 'request_id': 'req-ae8248fe-ac8b-43da-833a-8ea7554797b3'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:13.031910 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ae8248fe-ac8b-43da-833a-8ea7554797b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c6536154b424b40abb6a19fef51867c', 'request_id': 'req-ae8248fe-ac8b-43da-833a-8ea7554797b3'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:13.052470 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ae8248fe-ac8b-43da-833a-8ea7554797b3 admin admin] The token cache is being invalidate because project 3c6536154b424b40abb6a19fef51867c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:50:13.052797 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ae8248fe-ac8b-43da-833a-8ea7554797b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:13.054193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ae8248fe-ac8b-43da-833a-8ea7554797b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:13.057082 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1188/2345] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:50:12 2022] DELETE /identity/v3/projects/3c6536154b424b40abb6a19fef51867c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:50:14.794898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6bcab-83d4-45e6-873f-0bba97561b6d service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:14.797528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-31e6bcab-83d4-45e6-873f-0bba97561b6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:14.827398 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6bcab-83d4-45e6-873f-0bba97561b6d service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:14.879769 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:50:15.015901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c5c49-edce-453c-89fe-d6254a3517c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:15.017485 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c5c49-edce-453c-89fe-d6254a3517c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:15.020470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c5c49-edce-453c-89fe-d6254a3517c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:15.020968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c5c49-edce-453c-89fe-d6254a3517c8 admin admin] PATH_INFO: `/v3/users/8fc98b9faa9547109246ee8fb7854795` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:15.034370 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c5c49-edce-453c-89fe-d6254a3517c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fc98b9faa9547109246ee8fb7854795)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:15.034370 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c5c49-edce-453c-89fe-d6254a3517c8 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:15.148631 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-463c5c49-edce-453c-89fe-d6254a3517c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fc98b9faa9547109246ee8fb7854795'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:15.183560 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-463c5c49-edce-453c-89fe-d6254a3517c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fc98b9faa9547109246ee8fb7854795'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:15.200148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-463c5c49-edce-453c-89fe-d6254a3517c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fc98b9faa9547109246ee8fb7854795'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:15.200961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-463c5c49-edce-453c-89fe-d6254a3517c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fc98b9faa9547109246ee8fb7854795'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:15.201864 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-463c5c49-edce-453c-89fe-d6254a3517c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fc98b9faa9547109246ee8fb7854795'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:15.223515 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1158/2346] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:50:14 2022] DELETE /identity/v3/users/8fc98b9faa9547109246ee8fb7854795 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:50:15.230026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf33f27-5483-461f-8e9f-fe3205a1d6fa None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:15.230566 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf33f27-5483-461f-8e9f-fe3205a1d6fa None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:15.231404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf33f27-5483-461f-8e9f-fe3205a1d6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:15.257895 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-2bf33f27-5483-461f-8e9f-fe3205a1d6fa None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:50:15.259885 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2bf33f27-5483-461f-8e9f-fe3205a1d6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:15.397092 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1189/2347] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:50:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:50:15.406823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c5c49-edce-453c-89fe-d6254a3517c8 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:15.408506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-463c5c49-edce-453c-89fe-d6254a3517c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:15.497191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c5c49-edce-453c-89fe-d6254a3517c8 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:15.510725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4826748-e8aa-45ab-97f6-5be5acba4194 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:15.511592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4826748-e8aa-45ab-97f6-5be5acba4194 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:15.511963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4826748-e8aa-45ab-97f6-5be5acba4194 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:15.512341 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4826748-e8aa-45ab-97f6-5be5acba4194 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:15.529911 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4826748-e8aa-45ab-97f6-5be5acba4194 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:15.532746 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4826748-e8aa-45ab-97f6-5be5acba4194 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:15.548996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1159/2348] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:50:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:50:16.470392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf33f27-5483-461f-8e9f-fe3205a1d6fa None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:16.473238 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2bf33f27-5483-461f-8e9f-fe3205a1d6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:16.545235 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf33f27-5483-461f-8e9f-fe3205a1d6fa None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:16.557681 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed652579-0577-4980-910a-806d6ce97ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:16.557681 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed652579-0577-4980-910a-806d6ce97ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:16.558041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed652579-0577-4980-910a-806d6ce97ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:16.559159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed652579-0577-4980-910a-806d6ce97ce4 admin admin] PATH_INFO: `/v3/projects/7c41ab6121304ec491345e780431535a` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:16.560046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed652579-0577-4980-910a-806d6ce97ce4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c41ab6121304ec491345e780431535a)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:16.562309 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed652579-0577-4980-910a-806d6ce97ce4 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:16.630806 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:50:16.728623 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ed652579-0577-4980-910a-806d6ce97ce4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c41ab6121304ec491345e780431535a', 'request_id': 'req-ed652579-0577-4980-910a-806d6ce97ce4'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:16.743846 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ed652579-0577-4980-910a-806d6ce97ce4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c41ab6121304ec491345e780431535a', 'request_id': 'req-ed652579-0577-4980-910a-806d6ce97ce4'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:16.760339 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ed652579-0577-4980-910a-806d6ce97ce4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c41ab6121304ec491345e780431535a', 'request_id': 'req-ed652579-0577-4980-910a-806d6ce97ce4'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:16.787988 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ed652579-0577-4980-910a-806d6ce97ce4 admin admin] The token cache is being invalidate because project 7c41ab6121304ec491345e780431535a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:50:16.788340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ed652579-0577-4980-910a-806d6ce97ce4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:16.789031 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ed652579-0577-4980-910a-806d6ce97ce4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:50:16.797313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b94d7-5da4-4ed0-8fa6-4b0d465216cd None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:16.798657 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1190/2349] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:50:16 2022] DELETE /identity/v3/projects/7c41ab6121304ec491345e780431535a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:50:16.798979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b94d7-5da4-4ed0-8fa6-4b0d465216cd None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:16.799276 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b94d7-5da4-4ed0-8fa6-4b0d465216cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:16.864937 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-ec5b94d7-5da4-4ed0-8fa6-4b0d465216cd None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:50:16.866622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-ec5b94d7-5da4-4ed0-8fa6-4b0d465216cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:16.948302 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:50:17.039374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1160/2350] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:50:17.053267 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806b73f-8d1a-4c37-89fb-0f3da2c4dbc8 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:17.053522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806b73f-8d1a-4c37-89fb-0f3da2c4dbc8 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:17.053816 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806b73f-8d1a-4c37-89fb-0f3da2c4dbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:17.079674 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-9806b73f-8d1a-4c37-89fb-0f3da2c4dbc8 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:50:17.083381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-9806b73f-8d1a-4c37-89fb-0f3da2c4dbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:17.092113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1191/2351] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:50:17.097317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b94d7-5da4-4ed0-8fa6-4b0d465216cd None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:17.114068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b94d7-5da4-4ed0-8fa6-4b0d465216cd None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:17.129021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19c973-d779-4f64-adb3-f1b1962a2cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:17.129253 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19c973-d779-4f64-adb3-f1b1962a2cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:17.129611 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19c973-d779-4f64-adb3-f1b1962a2cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:17.130040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19c973-d779-4f64-adb3-f1b1962a2cb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:17.131124 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19c973-d779-4f64-adb3-f1b1962a2cb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:50:17.133687 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19c973-d779-4f64-adb3-f1b1962a2cb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:17.141126 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19c973-d779-4f64-adb3-f1b1962a2cb6 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:17.192175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806b73f-8d1a-4c37-89fb-0f3da2c4dbc8 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:17.203655 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1161/2352] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:50:17.205092 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806b73f-8d1a-4c37-89fb-0f3da2c4dbc8 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:17.214199 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d32a6-0890-43b4-93cc-123a678c5c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:17.215133 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d32a6-0890-43b4-93cc-123a678c5c53 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:17.215420 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d32a6-0890-43b4-93cc-123a678c5c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:17.216375 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d32a6-0890-43b4-93cc-123a678c5c53 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:17.217173 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d32a6-0890-43b4-93cc-123a678c5c53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:17.219973 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d32a6-0890-43b4-93cc-123a678c5c53 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:17.274423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19c973-d779-4f64-adb3-f1b1962a2cb6 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:17.287320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1192/2353] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:50:17 2022] POST /identity/v3/projects => generated 371 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:50:17.291964 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19c973-d779-4f64-adb3-f1b1962a2cb6 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:17.418286 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb1268-b029-467d-a858-accf160984a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:17.418475 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb1268-b029-467d-a858-accf160984a4 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:17.418904 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb1268-b029-467d-a858-accf160984a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:17.419189 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb1268-b029-467d-a858-accf160984a4 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:17.419897 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb1268-b029-467d-a858-accf160984a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:17.422239 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb1268-b029-467d-a858-accf160984a4 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:17.453872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1162/2354] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:50:17 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:50:17.458859 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d32a6-0890-43b4-93cc-123a678c5c53 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:17.485922 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d32a6-0890-43b4-93cc-123a678c5c53 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:17.502108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d8871-2224-4e3f-960d-de270138cfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:17.503332 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d8871-2224-4e3f-960d-de270138cfe1 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:17.503695 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d8871-2224-4e3f-960d-de270138cfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:17.504166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d8871-2224-4e3f-960d-de270138cfe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:17.505102 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d8871-2224-4e3f-960d-de270138cfe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:17.506981 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d8871-2224-4e3f-960d-de270138cfe1 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:17.535604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1193/2355] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:50:17.540338 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb1268-b029-467d-a858-accf160984a4 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:17.559800 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb1268-b029-467d-a858-accf160984a4 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:17.579359 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7ab7d1-ea50-4b81-96a8-eb47857b0fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:17.580067 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ab7d1-ea50-4b81-96a8-eb47857b0fdd admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:17.580367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ab7d1-ea50-4b81-96a8-eb47857b0fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:17.580644 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ab7d1-ea50-4b81-96a8-eb47857b0fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:17.582778 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ab7d1-ea50-4b81-96a8-eb47857b0fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:17.584431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7ab7d1-ea50-4b81-96a8-eb47857b0fdd admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:17.609907 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1163/2356] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:50:17.614841 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d8871-2224-4e3f-960d-de270138cfe1 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:17.632849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d8871-2224-4e3f-960d-de270138cfe1 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:17.644918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb96a77-8d45-40d0-adac-15e334fe682e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:17.645763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb96a77-8d45-40d0-adac-15e334fe682e admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:17.646060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb96a77-8d45-40d0-adac-15e334fe682e admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:17.646339 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb96a77-8d45-40d0-adac-15e334fe682e admin admin] PATH_INFO: `/v3/projects/74b7ceb2cdb24fe0b504de90f017ce0f/users/0c60a904281841fc8773877812f0b8bc/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:17.709241 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb96a77-8d45-40d0-adac-15e334fe682e admin admin] RBAC: Authorizing `identity:create_grant(project_id=74b7ceb2cdb24fe0b504de90f017ce0f, user_id=0c60a904281841fc8773877812f0b8bc, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:17.716106 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb96a77-8d45-40d0-adac-15e334fe682e admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:17.754255 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1194/2357] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:50:17 2022] PUT /identity/v3/projects/74b7ceb2cdb24fe0b504de90f017ce0f/users/0c60a904281841fc8773877812f0b8bc/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:50:17.756226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da8dc6-77d4-479e-ab88-df0c6f31f989 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:17.756522 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da8dc6-77d4-479e-ab88-df0c6f31f989 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:17.756805 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da8dc6-77d4-479e-ab88-df0c6f31f989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:17.792992 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-45da8dc6-77d4-479e-ab88-df0c6f31f989 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:50:17.795495 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-45da8dc6-77d4-479e-ab88-df0c6f31f989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:17.913761 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1164/2358] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:50:17.920349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb96a77-8d45-40d0-adac-15e334fe682e admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:17.922525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-4cb96a77-8d45-40d0-adac-15e334fe682e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:18.048082 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb96a77-8d45-40d0-adac-15e334fe682e admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:18.054380 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87a972-0f3e-4844-ae0f-fc9b031f5329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:18.054784 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87a972-0f3e-4844-ae0f-fc9b031f5329 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:18.055170 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87a972-0f3e-4844-ae0f-fc9b031f5329 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:18.055552 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87a972-0f3e-4844-ae0f-fc9b031f5329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:18.075351 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b87a972-0f3e-4844-ae0f-fc9b031f5329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:18.079323 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b87a972-0f3e-4844-ae0f-fc9b031f5329 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:18.094797 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1195/2359] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:50:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:50:19.313303 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a2b51-7419-40aa-9306-b49db17dd105 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:19.313937 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a2b51-7419-40aa-9306-b49db17dd105 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:19.313937 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a2b51-7419-40aa-9306-b49db17dd105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:19.420796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-5d3a2b51-7419-40aa-9306-b49db17dd105 None None] MFA Rules not processed for user `0c60a904281841fc8773877812f0b8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:50:19.422607 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-5d3a2b51-7419-40aa-9306-b49db17dd105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:19.480224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:50:19.545566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1165/2360] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:50:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:50:19.557776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87a972-0f3e-4844-ae0f-fc9b031f5329 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:19.559409 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0b87a972-0f3e-4844-ae0f-fc9b031f5329 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:19.642398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:50:19.648439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87a972-0f3e-4844-ae0f-fc9b031f5329 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:19.652286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4ec7c-847c-418d-bde9-e5c15b43bd9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:19.652977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4ec7c-847c-418d-bde9-e5c15b43bd9d service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:19.653263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4ec7c-847c-418d-bde9-e5c15b43bd9d service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:19.653542 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4ec7c-847c-418d-bde9-e5c15b43bd9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:19.676900 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4ec7c-847c-418d-bde9-e5c15b43bd9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:19.679562 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4ec7c-847c-418d-bde9-e5c15b43bd9d service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:19.708584 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1196/2361] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:50:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:50:20.351303 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ffda6-b0db-4904-891a-352a5451b322 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:20.351879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ffda6-b0db-4904-891a-352a5451b322 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:20.351879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ffda6-b0db-4904-891a-352a5451b322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:20.389350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-766ffda6-b0db-4904-891a-352a5451b322 None None] MFA Rules not processed for user `bf539804a8cf46ada01d8dea4fb1c09f`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:50:20.390669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-766ffda6-b0db-4904-891a-352a5451b322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:20.486135 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1166/2362] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:50:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:50:20.501456 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4ec7c-847c-418d-bde9-e5c15b43bd9d service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:50:20.503622 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-3ba4ec7c-847c-418d-bde9-e5c15b43bd9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:20.593386 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4ec7c-847c-418d-bde9-e5c15b43bd9d service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:50:20.599537 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da415414-9243-43ee-bb55-6297168c3abb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:50:20.600188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da415414-9243-43ee-bb55-6297168c3abb service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:20.600472 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da415414-9243-43ee-bb55-6297168c3abb service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:20.600751 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da415414-9243-43ee-bb55-6297168c3abb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:20.602536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-da415414-9243-43ee-bb55-6297168c3abb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:20.673687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da415414-9243-43ee-bb55-6297168c3abb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:50:20.673687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da415414-9243-43ee-bb55-6297168c3abb service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:50:20.686535 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1197/2363] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:50:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:50:21.238015 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489dc91-8d5d-4c87-898a-c0718b33d466 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:21.238367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489dc91-8d5d-4c87-898a-c0718b33d466 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:21.238645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489dc91-8d5d-4c87-898a-c0718b33d466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:21.259411 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-4489dc91-8d5d-4c87-898a-c0718b33d466 None None] MFA Rules not processed for user `0c60a904281841fc8773877812f0b8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:50:21.260712 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4489dc91-8d5d-4c87-898a-c0718b33d466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:21.269089 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1167/2364] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:50:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:50:30.680718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=60 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:50:31.252333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=52 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:50:31.754656 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca30266-fa49-4fae-8428-fda539323d84 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:31.755284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca30266-fa49-4fae-8428-fda539323d84 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:31.755284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca30266-fa49-4fae-8428-fda539323d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:31.776119 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-3ca30266-fa49-4fae-8428-fda539323d84 None None] MFA Rules not processed for user `1a7a81da8acf40849a9ea86e2542ae6a`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:50:31.777342 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-3ca30266-fa49-4fae-8428-fda539323d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:31.841557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1198/2365] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:50:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:50:41.834687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=25 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:50:45.660309 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5de33-f31f-4475-8dc8-7a09d4246a9a None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:45.660864 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5de33-f31f-4475-8dc8-7a09d4246a9a None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:45.660961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5de33-f31f-4475-8dc8-7a09d4246a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:45.689076 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-a6a5de33-f31f-4475-8dc8-7a09d4246a9a None None] MFA Rules not processed for user `25024a505319480ab00f4c7e58de600c`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:50:45.697146 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a6a5de33-f31f-4475-8dc8-7a09d4246a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:45.706295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1168/2366] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:50:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:50:55.606003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88956555-7115-42e6-b133-e878dc3c4fa0 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:50:55.606541 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88956555-7115-42e6-b133-e878dc3c4fa0 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:50:55.606639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88956555-7115-42e6-b133-e878dc3c4fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:50:55.630807 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-88956555-7115-42e6-b133-e878dc3c4fa0 None None] MFA Rules not processed for user `0c60a904281841fc8773877812f0b8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:50:55.632156 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-88956555-7115-42e6-b133-e878dc3c4fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:50:55.640944 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1199/2367] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:50:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:50:55.676491 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:51:05.627420 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:51:05.840400 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69402858-1ab2-4232-9838-0dfd673ea62c None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:05.840949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69402858-1ab2-4232-9838-0dfd673ea62c None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:05.840949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69402858-1ab2-4232-9838-0dfd673ea62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:05.860450 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-69402858-1ab2-4232-9838-0dfd673ea62c None None] MFA Rules not processed for user `7432931bd04d45bcaee686df05d5c5b3`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:51:05.863079 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-69402858-1ab2-4232-9838-0dfd673ea62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:51:05.922423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1169/2368] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:51:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:51:07.484393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88956555-7115-42e6-b133-e878dc3c4fa0 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:51:07.485984 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-88956555-7115-42e6-b133-e878dc3c4fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:51:07.603138 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88956555-7115-42e6-b133-e878dc3c4fa0 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:51:07.608576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb20da-62ee-4c61-95c6-bc11c005b2ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffafb68f1bb941588a5217ad83ad907c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:51:07.609300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb20da-62ee-4c61-95c6-bc11c005b2ab service placement] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:07.609586 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb20da-62ee-4c61-95c6-bc11c005b2ab service placement] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:07.609898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb20da-62ee-4c61-95c6-bc11c005b2ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:07.611802 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-5fdb20da-62ee-4c61-95c6-bc11c005b2ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:51:07.623918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb20da-62ee-4c61-95c6-bc11c005b2ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:51:07.625523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb20da-62ee-4c61-95c6-bc11c005b2ab service placement] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:51:07.639192 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1200/2369] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:51:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:51:15.915301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=25 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:51:16.632036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69402858-1ab2-4232-9838-0dfd673ea62c None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:51:16.633614 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-69402858-1ab2-4232-9838-0dfd673ea62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:51:16.647968 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69402858-1ab2-4232-9838-0dfd673ea62c None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:51:16.652108 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63d55e-03a5-47ba-ba04-29177121b326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:51:16.652851 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63d55e-03a5-47ba-ba04-29177121b326 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:16.653196 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63d55e-03a5-47ba-ba04-29177121b326 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:16.653525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63d55e-03a5-47ba-ba04-29177121b326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:16.665615 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63d55e-03a5-47ba-ba04-29177121b326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:51:16.667469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63d55e-03a5-47ba-ba04-29177121b326 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:51:16.680030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1170/2370] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:51:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:51:17.634666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=41 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:51:26.674771 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=6 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:51:29.164206 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215aba9-4a8f-4fef-89ec-60830989f7e8 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:29.164896 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215aba9-4a8f-4fef-89ec-60830989f7e8 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:29.165196 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215aba9-4a8f-4fef-89ec-60830989f7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:29.258957 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-0215aba9-4a8f-4fef-89ec-60830989f7e8 None None] MFA Rules not processed for user `6655366e7eb84a26aa3f7fbe88af62b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:51:29.260226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0215aba9-4a8f-4fef-89ec-60830989f7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:51:29.374319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1201/2371] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jun 27 15:51:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:51:29.443485 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63d55e-03a5-47ba-ba04-29177121b326 service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:51:29.445210 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-ca63d55e-03a5-47ba-ba04-29177121b326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:51:29.567775 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63d55e-03a5-47ba-ba04-29177121b326 service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:51:29.573426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb47168-0548-4aba-9109-1b8a2b100481 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e83ea69ded040448fe75caa221ad377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:51:29.574250 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb47168-0548-4aba-9109-1b8a2b100481 service swift] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:29.574642 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb47168-0548-4aba-9109-1b8a2b100481 service swift] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:29.575010 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb47168-0548-4aba-9109-1b8a2b100481 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:29.585915 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb47168-0548-4aba-9109-1b8a2b100481 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:51:29.587643 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb47168-0548-4aba-9109-1b8a2b100481 service swift] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:51:29.599032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1171/2372] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jun 27 15:51:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 15:51:39.368233 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=47 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:51:39.593987 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=41 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:51:41.161526 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16a952-5d43-4ac5-a8ae-80d6a9af69f5 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:41.162439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16a952-5d43-4ac5-a8ae-80d6a9af69f5 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:41.162439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16a952-5d43-4ac5-a8ae-80d6a9af69f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:41.201091 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-5e16a952-5d43-4ac5-a8ae-80d6a9af69f5 None None] MFA Rules not processed for user `6655366e7eb84a26aa3f7fbe88af62b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:51:41.202613 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-5e16a952-5d43-4ac5-a8ae-80d6a9af69f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:51:41.212148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1202/2373] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jun 27 15:51:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:51:41.233688 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb47168-0548-4aba-9109-1b8a2b100481 service swift] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:51:41.278931 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb47168-0548-4aba-9109-1b8a2b100481 service swift] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:51:41.296469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48620026-e82b-4ee9-abc2-e8c21d927860 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e83ea69ded040448fe75caa221ad377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:51:41.299644 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48620026-e82b-4ee9-abc2-e8c21d927860 service swift] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:41.300614 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48620026-e82b-4ee9-abc2-e8c21d927860 service swift] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:41.301460 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48620026-e82b-4ee9-abc2-e8c21d927860 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:41.324538 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48620026-e82b-4ee9-abc2-e8c21d927860 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:51:41.327201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48620026-e82b-4ee9-abc2-e8c21d927860 service swift] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:51:41.341849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1172/2374] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jun 27 15:51:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:51:49.961186 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b860a-3bab-4e87-88c8-ae94a564dcca None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:49.961859 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b860a-3bab-4e87-88c8-ae94a564dcca None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:49.962144 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b860a-3bab-4e87-88c8-ae94a564dcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:49.997980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-eb6b860a-3bab-4e87-88c8-ae94a564dcca None None] MFA Rules not processed for user `6655366e7eb84a26aa3f7fbe88af62b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:51:50.000290 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-eb6b860a-3bab-4e87-88c8-ae94a564dcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:51:50.012062 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1203/2375] 10.222.0.20 () {58 vars in 967 bytes} [Mon Jun 27 15:51:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:51:50.046553 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48620026-e82b-4ee9-abc2-e8c21d927860 service swift] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:51:50.067991 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48620026-e82b-4ee9-abc2-e8c21d927860 service swift] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:51:50.075481 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2254cc-d314-45c8-a9a5-89a7c2a783ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e83ea69ded040448fe75caa221ad377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:51:50.076162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2254cc-d314-45c8-a9a5-89a7c2a783ad service swift] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:50.076658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2254cc-d314-45c8-a9a5-89a7c2a783ad service swift] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:50.076936 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2254cc-d314-45c8-a9a5-89a7c2a783ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:50.093780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2254cc-d314-45c8-a9a5-89a7c2a783ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:51:50.096444 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2254cc-d314-45c8-a9a5-89a7c2a783ad service swift] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:51:50.113597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1173/2376] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jun 27 15:51:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 15:51:53.592761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b860a-3bab-4e87-88c8-ae94a564dcca None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:51:53.620898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b860a-3bab-4e87-88c8-ae94a564dcca None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:51:53.648291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117fcb7-4f00-4260-b6f8-d9b0be6b24e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:51:53.649210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117fcb7-4f00-4260-b6f8-d9b0be6b24e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:53.649485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117fcb7-4f00-4260-b6f8-d9b0be6b24e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:53.649985 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117fcb7-4f00-4260-b6f8-d9b0be6b24e1 admin admin] PATH_INFO: `/v3/users/0c60a904281841fc8773877812f0b8bc` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:53.652322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117fcb7-4f00-4260-b6f8-d9b0be6b24e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c60a904281841fc8773877812f0b8bc)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:51:53.654720 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117fcb7-4f00-4260-b6f8-d9b0be6b24e1 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:51:53.727598 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4117fcb7-4f00-4260-b6f8-d9b0be6b24e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c60a904281841fc8773877812f0b8bc'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:51:53.754802 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4117fcb7-4f00-4260-b6f8-d9b0be6b24e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c60a904281841fc8773877812f0b8bc'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:51:53.755660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4117fcb7-4f00-4260-b6f8-d9b0be6b24e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c60a904281841fc8773877812f0b8bc'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:51:53.769831 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4117fcb7-4f00-4260-b6f8-d9b0be6b24e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c60a904281841fc8773877812f0b8bc'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:51:53.783997 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4117fcb7-4f00-4260-b6f8-d9b0be6b24e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c60a904281841fc8773877812f0b8bc'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:51:53.795954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1204/2377] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:51:53 2022] DELETE /identity/v3/users/0c60a904281841fc8773877812f0b8bc => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:51:54.087513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2254cc-d314-45c8-a9a5-89a7c2a783ad service swift] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:51:54.105198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2254cc-d314-45c8-a9a5-89a7c2a783ad service swift] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:51:54.207779 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487c82b-2507-4c79-97d1-b62f1b565cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:51:54.210189 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487c82b-2507-4c79-97d1-b62f1b565cfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:54.210189 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487c82b-2507-4c79-97d1-b62f1b565cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:54.210646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487c82b-2507-4c79-97d1-b62f1b565cfa admin admin] PATH_INFO: `/v3/projects/74b7ceb2cdb24fe0b504de90f017ce0f` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:54.213901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487c82b-2507-4c79-97d1-b62f1b565cfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=74b7ceb2cdb24fe0b504de90f017ce0f)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:51:54.223215 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487c82b-2507-4c79-97d1-b62f1b565cfa admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:51:54.295300 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-0487c82b-2507-4c79-97d1-b62f1b565cfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74b7ceb2cdb24fe0b504de90f017ce0f', 'request_id': 'req-0487c82b-2507-4c79-97d1-b62f1b565cfa'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:51:54.309621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-0487c82b-2507-4c79-97d1-b62f1b565cfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74b7ceb2cdb24fe0b504de90f017ce0f', 'request_id': 'req-0487c82b-2507-4c79-97d1-b62f1b565cfa'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:51:54.320083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-0487c82b-2507-4c79-97d1-b62f1b565cfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74b7ceb2cdb24fe0b504de90f017ce0f', 'request_id': 'req-0487c82b-2507-4c79-97d1-b62f1b565cfa'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:51:54.341033 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-0487c82b-2507-4c79-97d1-b62f1b565cfa admin admin] The token cache is being invalidate because project 74b7ceb2cdb24fe0b504de90f017ce0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:51:54.341333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-0487c82b-2507-4c79-97d1-b62f1b565cfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:51:54.341958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-0487c82b-2507-4c79-97d1-b62f1b565cfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:51:54.343792 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1174/2378] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:51:54 2022] DELETE /identity/v3/projects/74b7ceb2cdb24fe0b504de90f017ce0f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:51:54.351972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992bb5b-d1a3-4dc6-b72b-fe3540112df3 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:54.352363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992bb5b-d1a3-4dc6-b72b-fe3540112df3 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:54.353036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992bb5b-d1a3-4dc6-b72b-fe3540112df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:54.392194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-7992bb5b-d1a3-4dc6-b72b-fe3540112df3 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:51:54.393620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-7992bb5b-d1a3-4dc6-b72b-fe3540112df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:51:54.487115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1205/2379] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:51:54.496177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246210e-8eb6-47ab-8e30-acbfc6c7b076 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:54.496497 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246210e-8eb6-47ab-8e30-acbfc6c7b076 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:54.496745 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246210e-8eb6-47ab-8e30-acbfc6c7b076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:54.519410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-9246210e-8eb6-47ab-8e30-acbfc6c7b076 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:51:54.521197 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-9246210e-8eb6-47ab-8e30-acbfc6c7b076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:51:54.542978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1175/2380] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:51:54.544215 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992bb5b-d1a3-4dc6-b72b-fe3540112df3 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:51:54.556745 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992bb5b-d1a3-4dc6-b72b-fe3540112df3 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:51:54.563084 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52e9c4-be18-4e9f-b8a6-aea96ceb5f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:51:54.563802 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52e9c4-be18-4e9f-b8a6-aea96ceb5f12 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:54.564153 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52e9c4-be18-4e9f-b8a6-aea96ceb5f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:54.564350 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52e9c4-be18-4e9f-b8a6-aea96ceb5f12 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:54.565177 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52e9c4-be18-4e9f-b8a6-aea96ceb5f12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:51:54.565493 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52e9c4-be18-4e9f-b8a6-aea96ceb5f12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:51:54.567389 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52e9c4-be18-4e9f-b8a6-aea96ceb5f12 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:51:54.581587 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1206/2381] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:51:54.588434 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246210e-8eb6-47ab-8e30-acbfc6c7b076 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:51:54.601377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246210e-8eb6-47ab-8e30-acbfc6c7b076 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:51:54.607719 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cadc5e-e6aa-452e-8d20-afc836d84a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:51:54.608552 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cadc5e-e6aa-452e-8d20-afc836d84a42 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:54.608900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cadc5e-e6aa-452e-8d20-afc836d84a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:54.609823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cadc5e-e6aa-452e-8d20-afc836d84a42 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:54.610935 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cadc5e-e6aa-452e-8d20-afc836d84a42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:51:54.613755 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cadc5e-e6aa-452e-8d20-afc836d84a42 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:51:54.653461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1176/2382] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:51:54 2022] POST /identity/v3/projects => generated 369 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:51:54.660260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52e9c4-be18-4e9f-b8a6-aea96ceb5f12 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:51:54.679699 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52e9c4-be18-4e9f-b8a6-aea96ceb5f12 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:51:54.742030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:51:54.774956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1588a-3e7b-4aa2-82c0-fb641e54258a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:51:54.775666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1588a-3e7b-4aa2-82c0-fb641e54258a admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:54.775955 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1588a-3e7b-4aa2-82c0-fb641e54258a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:54.776273 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1588a-3e7b-4aa2-82c0-fb641e54258a admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:54.776995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c1588a-3e7b-4aa2-82c0-fb641e54258a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:51:54.779084 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c1588a-3e7b-4aa2-82c0-fb641e54258a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:51:54.808304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1207/2383] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:51:54 2022] POST /identity/v3/users => generated 285 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:51:54.815585 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cadc5e-e6aa-452e-8d20-afc836d84a42 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:51:54.836385 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cadc5e-e6aa-452e-8d20-afc836d84a42 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:51:54.846643 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c06c5-b542-4619-9fb8-fbcaac2ce6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:51:54.847776 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c06c5-b542-4619-9fb8-fbcaac2ce6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:54.848310 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c06c5-b542-4619-9fb8-fbcaac2ce6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:54.848772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c06c5-b542-4619-9fb8-fbcaac2ce6a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:54.849979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c06c5-b542-4619-9fb8-fbcaac2ce6a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:51:54.852367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c06c5-b542-4619-9fb8-fbcaac2ce6a1 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:51:54.877721 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1177/2384] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:51:54.883899 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1588a-3e7b-4aa2-82c0-fb641e54258a admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:51:54.901010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1588a-3e7b-4aa2-82c0-fb641e54258a admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:51:54.911207 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46ec4b-1443-4631-8a77-353ccc843d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:51:54.912477 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46ec4b-1443-4631-8a77-353ccc843d2f admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:54.912956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46ec4b-1443-4631-8a77-353ccc843d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:54.913482 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46ec4b-1443-4631-8a77-353ccc843d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:54.914797 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46ec4b-1443-4631-8a77-353ccc843d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:51:54.918041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46ec4b-1443-4631-8a77-353ccc843d2f admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:51:54.938331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1208/2385] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:51:54.944314 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c06c5-b542-4619-9fb8-fbcaac2ce6a1 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:51:54.964745 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c06c5-b542-4619-9fb8-fbcaac2ce6a1 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:51:54.973408 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733f3dc-4483-4daf-9fb6-2150b74ec093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:51:54.974755 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733f3dc-4483-4daf-9fb6-2150b74ec093 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:54.975277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733f3dc-4483-4daf-9fb6-2150b74ec093 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:54.975659 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733f3dc-4483-4daf-9fb6-2150b74ec093 admin admin] PATH_INFO: `/v3/projects/4a34a388fa564791a612d638a955e4c9/users/97f8026120324e4c81a1ef7421718b8e/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:55.010017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1733f3dc-4483-4daf-9fb6-2150b74ec093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a34a388fa564791a612d638a955e4c9, user_id=97f8026120324e4c81a1ef7421718b8e, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:51:55.013810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1733f3dc-4483-4daf-9fb6-2150b74ec093 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:51:55.031670 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:51:55.043413 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1178/2386] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:51:54 2022] PUT /identity/v3/projects/4a34a388fa564791a612d638a955e4c9/users/97f8026120324e4c81a1ef7421718b8e/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:51:55.065583 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc2679-986b-4e21-8429-d2caa257bd89 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:55.066151 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc2679-986b-4e21-8429-d2caa257bd89 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:55.066634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc2679-986b-4e21-8429-d2caa257bd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:55.114389 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-48dc2679-986b-4e21-8429-d2caa257bd89 None None] MFA Rules not processed for user `97f8026120324e4c81a1ef7421718b8e`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:51:55.116027 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-48dc2679-986b-4e21-8429-d2caa257bd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:51:55.303107 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1209/2387] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:51:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:51:55.322992 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733f3dc-4483-4daf-9fb6-2150b74ec093 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:51:55.325102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1733f3dc-4483-4daf-9fb6-2150b74ec093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:51:55.471602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733f3dc-4483-4daf-9fb6-2150b74ec093 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:51:55.478332 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82573a0-fab3-4333-b4a9-5cc604f2be48 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493fcde4a45944efbc0b52c44e61b411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:51:55.479497 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82573a0-fab3-4333-b4a9-5cc604f2be48 service glance] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:51:55.480007 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82573a0-fab3-4333-b4a9-5cc604f2be48 service glance] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:51:55.480543 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82573a0-fab3-4333-b4a9-5cc604f2be48 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:51:55.498347 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82573a0-fab3-4333-b4a9-5cc604f2be48 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:51:55.502012 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82573a0-fab3-4333-b4a9-5cc604f2be48 service glance] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:51:55.517178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1179/2388] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:51:55 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:52:02.292739 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c62a3-8aa5-4ed9-bc0e-aef520de7380 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:02.294385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c62a3-8aa5-4ed9-bc0e-aef520de7380 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:02.295351 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c62a3-8aa5-4ed9-bc0e-aef520de7380 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:02.300054 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1210/2389] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jun 27 15:52:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:52:02.311008 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ab1d7-f4f8-4e26-bf46-8e132fd1f03f None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:02.311645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ab1d7-f4f8-4e26-bf46-8e132fd1f03f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:02.312738 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ab1d7-f4f8-4e26-bf46-8e132fd1f03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:02.481614 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-1e6ab1d7-f4f8-4e26-bf46-8e132fd1f03f None None] MFA Rules not processed for user `1c8f6262cf24419daf803ae71aed24f5`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:02.486380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1e6ab1d7-f4f8-4e26-bf46-8e132fd1f03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:02.649044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1180/2390] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jun 27 15:52:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:52:02.665568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c62a3-8aa5-4ed9-bc0e-aef520de7380 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:02.682916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c62a3-8aa5-4ed9-bc0e-aef520de7380 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:02.689696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe6c86-83a5-4214-94dc-7a80baf0a5a0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493fcde4a45944efbc0b52c44e61b411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:02.691191 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe6c86-83a5-4214-94dc-7a80baf0a5a0 service glance] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:02.691545 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe6c86-83a5-4214-94dc-7a80baf0a5a0 service glance] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:02.691915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe6c86-83a5-4214-94dc-7a80baf0a5a0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:02.705314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe6c86-83a5-4214-94dc-7a80baf0a5a0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:02.707082 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe6c86-83a5-4214-94dc-7a80baf0a5a0 service glance] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:02.722103 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1211/2391] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:52:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:52:02.857545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf5072-dafa-46ff-9873-076dd1340849 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:02.858201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf5072-dafa-46ff-9873-076dd1340849 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:02.859575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf5072-dafa-46ff-9873-076dd1340849 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:02.861584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1181/2392] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jun 27 15:52:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:52:02.868243 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f846d1f-f0c3-4330-a090-8c4a71bb9f71 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:02.869157 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f846d1f-f0c3-4330-a090-8c4a71bb9f71 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:02.870251 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f846d1f-f0c3-4330-a090-8c4a71bb9f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:02.897966 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-2f846d1f-f0c3-4330-a090-8c4a71bb9f71 None None] MFA Rules not processed for user `42dee639e2834d99b7468f911c37cf8f`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:02.899670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2f846d1f-f0c3-4330-a090-8c4a71bb9f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:03.004731 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1212/2393] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jun 27 15:52:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:52:03.020342 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf5072-dafa-46ff-9873-076dd1340849 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:03.024066 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-cdcf5072-dafa-46ff-9873-076dd1340849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:03.040774 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf5072-dafa-46ff-9873-076dd1340849 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:03.045132 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d259ae8-781b-4c88-a2e1-9b1e4fd05fdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:03.045960 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d259ae8-781b-4c88-a2e1-9b1e4fd05fdd service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:03.047156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d259ae8-781b-4c88-a2e1-9b1e4fd05fdd service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:03.047503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d259ae8-781b-4c88-a2e1-9b1e4fd05fdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:03.056910 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d259ae8-781b-4c88-a2e1-9b1e4fd05fdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:03.058864 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d259ae8-781b-4c88-a2e1-9b1e4fd05fdd service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:03.068931 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1182/2394] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:52:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:52:03.182793 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cca520-7c4d-4edb-ae22-851405a85369 None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:03.183074 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cca520-7c4d-4edb-ae22-851405a85369 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:03.183393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cca520-7c4d-4edb-ae22-851405a85369 None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:03.185981 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1213/2395] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jun 27 15:52:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 15:52:03.204191 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff81ba1-ca82-45e5-a83b-034f989646c4 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:03.205030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff81ba1-ca82-45e5-a83b-034f989646c4 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:03.205549 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff81ba1-ca82-45e5-a83b-034f989646c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:03.232224 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-1ff81ba1-ca82-45e5-a83b-034f989646c4 None None] MFA Rules not processed for user `1c8f6262cf24419daf803ae71aed24f5`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:03.233502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1ff81ba1-ca82-45e5-a83b-034f989646c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:03.246274 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1183/2396] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jun 27 15:52:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:52:03.258980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cca520-7c4d-4edb-ae22-851405a85369 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:03.272585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cca520-7c4d-4edb-ae22-851405a85369 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:03.277233 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f8e17-7f4a-4244-9720-646fa306d125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:03.278522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f8e17-7f4a-4244-9720-646fa306d125 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:03.279151 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f8e17-7f4a-4244-9720-646fa306d125 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:03.279519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f8e17-7f4a-4244-9720-646fa306d125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:03.307158 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f8e17-7f4a-4244-9720-646fa306d125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:03.309490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f8e17-7f4a-4244-9720-646fa306d125 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:03.324539 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1214/2397] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:52:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:52:13.221362 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=90 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:52:13.318634 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:52:14.335175 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4b6bd-1450-42c2-aa9c-7a00874830f7 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:14.335750 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4b6bd-1450-42c2-aa9c-7a00874830f7 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:14.335750 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4b6bd-1450-42c2-aa9c-7a00874830f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:14.356229 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-4ef4b6bd-1450-42c2-aa9c-7a00874830f7 None None] MFA Rules not processed for user `6655366e7eb84a26aa3f7fbe88af62b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:14.357469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4ef4b6bd-1450-42c2-aa9c-7a00874830f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:14.449622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1184/2398] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jun 27 15:52:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:52:14.470108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f8e17-7f4a-4244-9720-646fa306d125 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:14.471719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-9c0f8e17-7f4a-4244-9720-646fa306d125 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:14.571838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f8e17-7f4a-4244-9720-646fa306d125 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:14.580015 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef68d9e-b8a3-4997-a03c-33c8a4ee5e1a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e83ea69ded040448fe75caa221ad377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:14.580703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef68d9e-b8a3-4997-a03c-33c8a4ee5e1a service swift] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:14.580984 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef68d9e-b8a3-4997-a03c-33c8a4ee5e1a service swift] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:14.581294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef68d9e-b8a3-4997-a03c-33c8a4ee5e1a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:14.591796 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef68d9e-b8a3-4997-a03c-33c8a4ee5e1a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:14.593497 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef68d9e-b8a3-4997-a03c-33c8a4ee5e1a service swift] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:14.603117 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1215/2399] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jun 27 15:52:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:52:24.436023 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=25 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:52:24.483961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4b6bd-1450-42c2-aa9c-7a00874830f7 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:24.485816 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4ef4b6bd-1450-42c2-aa9c-7a00874830f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:24.584153 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4b6bd-1450-42c2-aa9c-7a00874830f7 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:24.589152 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406d44e-347e-4b81-8d15-2e1446677fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:24.590345 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406d44e-347e-4b81-8d15-2e1446677fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:24.590768 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406d44e-347e-4b81-8d15-2e1446677fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:24.591150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406d44e-347e-4b81-8d15-2e1446677fb0 admin admin] PATH_INFO: `/v3/users/7432931bd04d45bcaee686df05d5c5b3` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:24.593260 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406d44e-347e-4b81-8d15-2e1446677fb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7432931bd04d45bcaee686df05d5c5b3)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:24.595703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406d44e-347e-4b81-8d15-2e1446677fb0 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:24.599146 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=32 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:52:24.661214 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a406d44e-347e-4b81-8d15-2e1446677fb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7432931bd04d45bcaee686df05d5c5b3'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:24.675974 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a406d44e-347e-4b81-8d15-2e1446677fb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7432931bd04d45bcaee686df05d5c5b3'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:24.685272 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a406d44e-347e-4b81-8d15-2e1446677fb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7432931bd04d45bcaee686df05d5c5b3'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:24.686421 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a406d44e-347e-4b81-8d15-2e1446677fb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7432931bd04d45bcaee686df05d5c5b3'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:24.687662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a406d44e-347e-4b81-8d15-2e1446677fb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7432931bd04d45bcaee686df05d5c5b3'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:24.712334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1185/2400] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:52:24 2022] DELETE /identity/v3/users/7432931bd04d45bcaee686df05d5c5b3 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:52:25.002452 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef68d9e-b8a3-4997-a03c-33c8a4ee5e1a service swift] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:25.022178 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef68d9e-b8a3-4997-a03c-33c8a4ee5e1a service swift] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:25.107513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d07d51-0cc6-4929-a4b8-5f7918276755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:25.109660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d07d51-0cc6-4929-a4b8-5f7918276755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:25.109971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d07d51-0cc6-4929-a4b8-5f7918276755 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:25.110256 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d07d51-0cc6-4929-a4b8-5f7918276755 admin admin] PATH_INFO: `/v3/projects/e552ed97892f41259a1d7057aea64f6b` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:25.111563 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d07d51-0cc6-4929-a4b8-5f7918276755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e552ed97892f41259a1d7057aea64f6b)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:25.113851 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d07d51-0cc6-4929-a4b8-5f7918276755 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:25.179113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-75d07d51-0cc6-4929-a4b8-5f7918276755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e552ed97892f41259a1d7057aea64f6b', 'request_id': 'req-75d07d51-0cc6-4929-a4b8-5f7918276755'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:25.189354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-75d07d51-0cc6-4929-a4b8-5f7918276755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e552ed97892f41259a1d7057aea64f6b', 'request_id': 'req-75d07d51-0cc6-4929-a4b8-5f7918276755'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:25.203257 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-75d07d51-0cc6-4929-a4b8-5f7918276755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e552ed97892f41259a1d7057aea64f6b', 'request_id': 'req-75d07d51-0cc6-4929-a4b8-5f7918276755'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:25.216472 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-75d07d51-0cc6-4929-a4b8-5f7918276755 admin admin] The token cache is being invalidate because project e552ed97892f41259a1d7057aea64f6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:52:25.216762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-75d07d51-0cc6-4929-a4b8-5f7918276755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:25.217559 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-75d07d51-0cc6-4929-a4b8-5f7918276755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:25.219331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1216/2401] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:52:25 2022] DELETE /identity/v3/projects/e552ed97892f41259a1d7057aea64f6b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:52:25.225385 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453170c-8463-4bdf-8738-fe6a57668767 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:25.225939 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453170c-8463-4bdf-8738-fe6a57668767 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:25.226420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453170c-8463-4bdf-8738-fe6a57668767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:25.257719 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-5453170c-8463-4bdf-8738-fe6a57668767 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:25.259745 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-5453170c-8463-4bdf-8738-fe6a57668767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:25.380636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1186/2402] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:52:25.459527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57af38-e31e-4015-b829-539d681da1b1 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:25.459812 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57af38-e31e-4015-b829-539d681da1b1 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:25.460091 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57af38-e31e-4015-b829-539d681da1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:25.480646 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-9a57af38-e31e-4015-b829-539d681da1b1 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:25.481937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-9a57af38-e31e-4015-b829-539d681da1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:25.490612 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1217/2403] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:52:25.495239 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453170c-8463-4bdf-8738-fe6a57668767 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:25.506697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453170c-8463-4bdf-8738-fe6a57668767 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:25.512065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d03fe4-726d-4d70-99c6-aa68d977d965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:25.512679 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d03fe4-726d-4d70-99c6-aa68d977d965 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:25.513158 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d03fe4-726d-4d70-99c6-aa68d977d965 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:25.513527 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d03fe4-726d-4d70-99c6-aa68d977d965 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:25.514576 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d03fe4-726d-4d70-99c6-aa68d977d965 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:52:25.514998 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d03fe4-726d-4d70-99c6-aa68d977d965 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:25.517464 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d03fe4-726d-4d70-99c6-aa68d977d965 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:25.536988 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1187/2404] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:52:25.542305 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57af38-e31e-4015-b829-539d681da1b1 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:25.553739 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57af38-e31e-4015-b829-539d681da1b1 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:25.558867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e80835-d56e-4cc1-ac71-ea2db05dc913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:25.559542 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e80835-d56e-4cc1-ac71-ea2db05dc913 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:25.559848 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e80835-d56e-4cc1-ac71-ea2db05dc913 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:25.560125 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e80835-d56e-4cc1-ac71-ea2db05dc913 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:25.560842 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e80835-d56e-4cc1-ac71-ea2db05dc913 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:25.562917 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e80835-d56e-4cc1-ac71-ea2db05dc913 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:25.599713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1218/2405] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:52:25 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:52:25.604283 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d03fe4-726d-4d70-99c6-aa68d977d965 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:25.615315 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d03fe4-726d-4d70-99c6-aa68d977d965 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:25.619520 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:52:25.693172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2972434-9b40-4ee1-9c38-802aa0018229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:25.694206 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2972434-9b40-4ee1-9c38-802aa0018229 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:25.694665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2972434-9b40-4ee1-9c38-802aa0018229 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:25.695079 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2972434-9b40-4ee1-9c38-802aa0018229 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:25.696915 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2972434-9b40-4ee1-9c38-802aa0018229 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:25.698893 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2972434-9b40-4ee1-9c38-802aa0018229 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:25.729546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1188/2406] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:52:25 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:52:25.734611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e80835-d56e-4cc1-ac71-ea2db05dc913 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:25.747718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e80835-d56e-4cc1-ac71-ea2db05dc913 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:25.754865 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4cd32-2090-444f-9c38-2fddd382bb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:25.756100 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4cd32-2090-444f-9c38-2fddd382bb27 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:25.756590 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4cd32-2090-444f-9c38-2fddd382bb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:25.757393 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4cd32-2090-444f-9c38-2fddd382bb27 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:25.759223 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4cd32-2090-444f-9c38-2fddd382bb27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:25.763028 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4cd32-2090-444f-9c38-2fddd382bb27 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:25.780706 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1219/2407] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:52:25.785328 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2972434-9b40-4ee1-9c38-802aa0018229 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:25.797596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2972434-9b40-4ee1-9c38-802aa0018229 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:25.803677 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73b27f-04c6-4211-b3bc-f91fcd3f9fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:25.804374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73b27f-04c6-4211-b3bc-f91fcd3f9fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:25.804660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73b27f-04c6-4211-b3bc-f91fcd3f9fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:25.804941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73b27f-04c6-4211-b3bc-f91fcd3f9fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:25.805728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e73b27f-04c6-4211-b3bc-f91fcd3f9fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:25.807368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e73b27f-04c6-4211-b3bc-f91fcd3f9fa4 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:25.820293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1189/2408] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:52:25.825528 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4cd32-2090-444f-9c38-2fddd382bb27 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:25.837697 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4cd32-2090-444f-9c38-2fddd382bb27 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:25.844031 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eacf534-cbf9-47cb-bbf1-cb764c032cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:25.845097 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eacf534-cbf9-47cb-bbf1-cb764c032cb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:25.845498 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eacf534-cbf9-47cb-bbf1-cb764c032cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:25.845928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eacf534-cbf9-47cb-bbf1-cb764c032cb0 admin admin] PATH_INFO: `/v3/projects/6a070d552fee4594bbcf0ae7adc16f94/users/1320328d4bd04dc28c0c828a0f30e641/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:25.863846 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eacf534-cbf9-47cb-bbf1-cb764c032cb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a070d552fee4594bbcf0ae7adc16f94, user_id=1320328d4bd04dc28c0c828a0f30e641, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:25.866748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eacf534-cbf9-47cb-bbf1-cb764c032cb0 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:25.883083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1220/2409] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:52:25 2022] PUT /identity/v3/projects/6a070d552fee4594bbcf0ae7adc16f94/users/1320328d4bd04dc28c0c828a0f30e641/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:52:25.889003 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df18e02-7e1f-49a8-8851-6b674656075d None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:25.890038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df18e02-7e1f-49a8-8851-6b674656075d None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:25.890409 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df18e02-7e1f-49a8-8851-6b674656075d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:25.911932 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-3df18e02-7e1f-49a8-8851-6b674656075d None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:25.913169 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3df18e02-7e1f-49a8-8851-6b674656075d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:25.981111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1190/2410] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:52:25.992582 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eacf534-cbf9-47cb-bbf1-cb764c032cb0 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:25.994614 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2eacf534-cbf9-47cb-bbf1-cb764c032cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:26.031666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:52:26.067807 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eacf534-cbf9-47cb-bbf1-cb764c032cb0 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:26.071660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e119b-9c7e-4974-a2b3-5395d1391984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:26.072330 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e119b-9c7e-4974-a2b3-5395d1391984 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:26.072618 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e119b-9c7e-4974-a2b3-5395d1391984 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:26.072910 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e119b-9c7e-4974-a2b3-5395d1391984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:26.084521 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e119b-9c7e-4974-a2b3-5395d1391984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:26.086060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e119b-9c7e-4974-a2b3-5395d1391984 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:26.097861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1221/2411] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:52:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:52:32.843440 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df18e02-7e1f-49a8-8851-6b674656075d None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:32.844628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3df18e02-7e1f-49a8-8851-6b674656075d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:32.867117 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df18e02-7e1f-49a8-8851-6b674656075d None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:32.883350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fdb65-4664-4a32-91cc-ce941092ccc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:32.885362 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fdb65-4664-4a32-91cc-ce941092ccc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:32.886237 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fdb65-4664-4a32-91cc-ce941092ccc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:32.886747 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780fdb65-4664-4a32-91cc-ce941092ccc3 admin admin] PATH_INFO: `/v3/users/97f8026120324e4c81a1ef7421718b8e` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:32.890580 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fdb65-4664-4a32-91cc-ce941092ccc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97f8026120324e4c81a1ef7421718b8e)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:32.898085 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780fdb65-4664-4a32-91cc-ce941092ccc3 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:33.022645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-780fdb65-4664-4a32-91cc-ce941092ccc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97f8026120324e4c81a1ef7421718b8e'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:33.057929 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-780fdb65-4664-4a32-91cc-ce941092ccc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97f8026120324e4c81a1ef7421718b8e'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:33.075530 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-780fdb65-4664-4a32-91cc-ce941092ccc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97f8026120324e4c81a1ef7421718b8e'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:33.077471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-780fdb65-4664-4a32-91cc-ce941092ccc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97f8026120324e4c81a1ef7421718b8e'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:33.078591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-780fdb65-4664-4a32-91cc-ce941092ccc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97f8026120324e4c81a1ef7421718b8e'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:33.100741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:52:33.115012 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1191/2412] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:52:32 2022] DELETE /identity/v3/users/97f8026120324e4c81a1ef7421718b8e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:52:33.117695 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9b60a-e58f-4032-a59a-54596baab3e4 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:33.118254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9b60a-e58f-4032-a59a-54596baab3e4 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:33.118701 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9b60a-e58f-4032-a59a-54596baab3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:33.144274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-e1b9b60a-e58f-4032-a59a-54596baab3e4 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:33.145770 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e1b9b60a-e58f-4032-a59a-54596baab3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:33.284166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1222/2413] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:52:33.301535 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fdb65-4664-4a32-91cc-ce941092ccc3 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:33.303844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-780fdb65-4664-4a32-91cc-ce941092ccc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:33.434560 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780fdb65-4664-4a32-91cc-ce941092ccc3 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:33.443128 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdaee80-5d13-4d8d-89de-d8ae661cb3c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:33.444784 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdaee80-5d13-4d8d-89de-d8ae661cb3c6 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:33.445359 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdaee80-5d13-4d8d-89de-d8ae661cb3c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:33.445958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdaee80-5d13-4d8d-89de-d8ae661cb3c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:33.465779 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdaee80-5d13-4d8d-89de-d8ae661cb3c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:33.468439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdaee80-5d13-4d8d-89de-d8ae661cb3c6 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:33.488877 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1192/2414] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:52:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:52:33.963173 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deec420-4fba-41cb-b913-d036950c98e0 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:33.963643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deec420-4fba-41cb-b913-d036950c98e0 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:33.964032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deec420-4fba-41cb-b913-d036950c98e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:34.023092 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-1deec420-4fba-41cb-b913-d036950c98e0 None None] MFA Rules not processed for user `1320328d4bd04dc28c0c828a0f30e641`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:34.025755 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1deec420-4fba-41cb-b913-d036950c98e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:34.127262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdaee80-5d13-4d8d-89de-d8ae661cb3c6 service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:34.129715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4bdaee80-5d13-4d8d-89de-d8ae661cb3c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:34.190019 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdaee80-5d13-4d8d-89de-d8ae661cb3c6 service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:34.198039 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50cbc3-9da0-4d45-af85-a23f2a014ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:34.199542 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50cbc3-9da0-4d45-af85-a23f2a014ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:34.199842 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50cbc3-9da0-4d45-af85-a23f2a014ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:34.200132 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50cbc3-9da0-4d45-af85-a23f2a014ea6 admin admin] PATH_INFO: `/v3/projects/4a34a388fa564791a612d638a955e4c9` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:34.203105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50cbc3-9da0-4d45-af85-a23f2a014ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a34a388fa564791a612d638a955e4c9)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:34.209301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50cbc3-9da0-4d45-af85-a23f2a014ea6 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:34.251184 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1223/2415] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:52:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:52:34.255882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deec420-4fba-41cb-b913-d036950c98e0 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:34.257559 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1deec420-4fba-41cb-b913-d036950c98e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:34.289965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-6e50cbc3-9da0-4d45-af85-a23f2a014ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a34a388fa564791a612d638a955e4c9', 'request_id': 'req-6e50cbc3-9da0-4d45-af85-a23f2a014ea6'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:34.303895 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deec420-4fba-41cb-b913-d036950c98e0 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:34.314605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-6e50cbc3-9da0-4d45-af85-a23f2a014ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a34a388fa564791a612d638a955e4c9', 'request_id': 'req-6e50cbc3-9da0-4d45-af85-a23f2a014ea6'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:34.341796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-6e50cbc3-9da0-4d45-af85-a23f2a014ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a34a388fa564791a612d638a955e4c9', 'request_id': 'req-6e50cbc3-9da0-4d45-af85-a23f2a014ea6'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:34.377865 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-6e50cbc3-9da0-4d45-af85-a23f2a014ea6 admin admin] The token cache is being invalidate because project 4a34a388fa564791a612d638a955e4c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:52:34.378746 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-6e50cbc3-9da0-4d45-af85-a23f2a014ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:34.380238 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-6e50cbc3-9da0-4d45-af85-a23f2a014ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:34.386644 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1193/2416] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:52:34 2022] DELETE /identity/v3/projects/4a34a388fa564791a612d638a955e4c9 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:52:34.393366 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393decc-9930-4894-a25e-5dd3b67d1fdd None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:34.396238 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393decc-9930-4894-a25e-5dd3b67d1fdd None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:34.396238 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393decc-9930-4894-a25e-5dd3b67d1fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:34.445627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b6b14-63bc-4857-9b47-56986994a7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:34.446600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b6b14-63bc-4857-9b47-56986994a7b9 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:34.446995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b6b14-63bc-4857-9b47-56986994a7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:34.447379 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b6b14-63bc-4857-9b47-56986994a7b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:34.448224 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b6b14-63bc-4857-9b47-56986994a7b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:34.451074 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b6b14-63bc-4857-9b47-56986994a7b9 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:34.457028 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-5393decc-9930-4894-a25e-5dd3b67d1fdd None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:34.457847 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:52:34.461014 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-5393decc-9930-4894-a25e-5dd3b67d1fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:34.486584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1194/2417] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:52:34.495969 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4575-8e79-4e1c-b047-9f213b6c5dac None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:34.496450 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4575-8e79-4e1c-b047-9f213b6c5dac None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:34.497057 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4575-8e79-4e1c-b047-9f213b6c5dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:34.503536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1224/2418] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:52:34 2022] POST /identity/v3/projects => generated 383 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:52:34.505060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b6b14-63bc-4857-9b47-56986994a7b9 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:34.507334 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-807b6b14-63bc-4857-9b47-56986994a7b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:34.557731 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-49aa4575-8e79-4e1c-b047-9f213b6c5dac None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:34.559108 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-49aa4575-8e79-4e1c-b047-9f213b6c5dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:34.674513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1195/2419] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:52:34.678669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa4575-8e79-4e1c-b047-9f213b6c5dac None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:34.685249 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b6b14-63bc-4857-9b47-56986994a7b9 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:34.695303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d36f7-acbf-42cb-a937-d75fb4efc747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:34.696195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa4575-8e79-4e1c-b047-9f213b6c5dac None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:34.696458 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d36f7-acbf-42cb-a937-d75fb4efc747 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:34.697609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d36f7-acbf-42cb-a937-d75fb4efc747 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:34.697609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d36f7-acbf-42cb-a937-d75fb4efc747 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:34.698778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d36f7-acbf-42cb-a937-d75fb4efc747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:34.702167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01228a-f4da-45aa-8540-d39401b72e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:34.703025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01228a-f4da-45aa-8540-d39401b72e57 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:34.703417 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01228a-f4da-45aa-8540-d39401b72e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:34.703797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01228a-f4da-45aa-8540-d39401b72e57 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:34.704145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d36f7-acbf-42cb-a937-d75fb4efc747 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:34.704873 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01228a-f4da-45aa-8540-d39401b72e57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:52:34.705291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01228a-f4da-45aa-8540-d39401b72e57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:34.707112 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01228a-f4da-45aa-8540-d39401b72e57 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:34.724681 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1196/2420] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:52:34.730132 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01228a-f4da-45aa-8540-d39401b72e57 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:34.737693 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:52:34.742560 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01228a-f4da-45aa-8540-d39401b72e57 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:34.756383 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1225/2421] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:52:34 2022] POST /identity/v3/users => generated 298 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:52:34.761205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ef69e-b6ab-4d5c-815d-458a9937584a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:34.762641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ef69e-b6ab-4d5c-815d-458a9937584a admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:34.762877 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d36f7-acbf-42cb-a937-d75fb4efc747 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:34.763172 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ef69e-b6ab-4d5c-815d-458a9937584a admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:34.763537 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ef69e-b6ab-4d5c-815d-458a9937584a admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:34.764920 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ef69e-b6ab-4d5c-815d-458a9937584a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:34.768716 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ef69e-b6ab-4d5c-815d-458a9937584a admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:34.780496 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d36f7-acbf-42cb-a937-d75fb4efc747 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:34.787646 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafb437-274f-451e-b346-3eb86f156ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:34.788345 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafb437-274f-451e-b346-3eb86f156ebd admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:34.788639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafb437-274f-451e-b346-3eb86f156ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:34.788914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafb437-274f-451e-b346-3eb86f156ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:34.789748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafb437-274f-451e-b346-3eb86f156ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:34.791414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafb437-274f-451e-b346-3eb86f156ebd admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:34.809788 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1226/2422] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:52:34.810616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1197/2423] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:52:34 2022] POST /identity/v3/projects => generated 375 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:52:34.814449 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafb437-274f-451e-b346-3eb86f156ebd admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:34.817382 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ef69e-b6ab-4d5c-815d-458a9937584a admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:34.826468 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafb437-274f-451e-b346-3eb86f156ebd admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:34.938232 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98102dae-1bb2-49f1-b1c6-0adadf5742f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:34.939786 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98102dae-1bb2-49f1-b1c6-0adadf5742f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:34.940405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98102dae-1bb2-49f1-b1c6-0adadf5742f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:34.940792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98102dae-1bb2-49f1-b1c6-0adadf5742f1 admin admin] PATH_INFO: `/v3/projects/4a048153141b485a92cfb09dd6902a91/users/7f551fe9c2d04de184de188eda5a830a/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:34.947734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ef69e-b6ab-4d5c-815d-458a9937584a admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:34.956265 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb0bb7-c3b1-4911-a156-f345f36e1183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:34.957190 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb0bb7-c3b1-4911-a156-f345f36e1183 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:34.957868 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb0bb7-c3b1-4911-a156-f345f36e1183 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:34.958299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb0bb7-c3b1-4911-a156-f345f36e1183 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:34.959696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb0bb7-c3b1-4911-a156-f345f36e1183 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:34.963543 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb0bb7-c3b1-4911-a156-f345f36e1183 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:34.968549 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98102dae-1bb2-49f1-b1c6-0adadf5742f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a048153141b485a92cfb09dd6902a91, user_id=7f551fe9c2d04de184de188eda5a830a, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:34.974655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98102dae-1bb2-49f1-b1c6-0adadf5742f1 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:34.994656 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1198/2424] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:52:34 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:52:34.995123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1227/2425] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:52:34 2022] PUT /identity/v3/projects/4a048153141b485a92cfb09dd6902a91/users/7f551fe9c2d04de184de188eda5a830a/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:52:35.001197 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb0bb7-c3b1-4911-a156-f345f36e1183 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:35.001456 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98102dae-1bb2-49f1-b1c6-0adadf5742f1 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:35.016425 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb0bb7-c3b1-4911-a156-f345f36e1183 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:35.087677 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:52:35.116456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e76db-7c02-459e-97e5-15c79467c40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:35.117605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e76db-7c02-459e-97e5-15c79467c40f admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:35.118814 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e76db-7c02-459e-97e5-15c79467c40f admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:35.119313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e76db-7c02-459e-97e5-15c79467c40f admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:35.120454 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98102dae-1bb2-49f1-b1c6-0adadf5742f1 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:35.120598 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e76db-7c02-459e-97e5-15c79467c40f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:35.123783 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e76db-7c02-459e-97e5-15c79467c40f admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:35.128801 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b2463-71e1-4bd6-babc-d60e2aa54b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:35.130107 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b2463-71e1-4bd6-babc-d60e2aa54b3b admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:35.130740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b2463-71e1-4bd6-babc-d60e2aa54b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:35.131385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b2463-71e1-4bd6-babc-d60e2aa54b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:35.132836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b2463-71e1-4bd6-babc-d60e2aa54b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:35.135767 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b2463-71e1-4bd6-babc-d60e2aa54b3b admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:35.143539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1199/2426] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:52:35.149948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e76db-7c02-459e-97e5-15c79467c40f admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:35.151384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1228/2427] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:52:35.157119 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b2463-71e1-4bd6-babc-d60e2aa54b3b admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:35.167638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b2463-71e1-4bd6-babc-d60e2aa54b3b admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:35.169092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e76db-7c02-459e-97e5-15c79467c40f admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:35.174187 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8580ed0-df72-4b63-9028-9f8af2014071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:35.175124 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8580ed0-df72-4b63-9028-9f8af2014071 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:35.175411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8580ed0-df72-4b63-9028-9f8af2014071 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:35.175692 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8580ed0-df72-4b63-9028-9f8af2014071 admin admin] PATH_INFO: `/v3/projects/4a048153141b485a92cfb09dd6902a91/users/7f551fe9c2d04de184de188eda5a830a/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:35.177852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8580ed0-df72-4b63-9028-9f8af2014071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a048153141b485a92cfb09dd6902a91, user_id=7f551fe9c2d04de184de188eda5a830a, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:35.178093 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584e79f-83a7-4d8c-b229-d27c48afba58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:35.179226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584e79f-83a7-4d8c-b229-d27c48afba58 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:35.179714 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584e79f-83a7-4d8c-b229-d27c48afba58 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:35.180205 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584e79f-83a7-4d8c-b229-d27c48afba58 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:35.180610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8580ed0-df72-4b63-9028-9f8af2014071 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:35.181736 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584e79f-83a7-4d8c-b229-d27c48afba58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:35.185004 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584e79f-83a7-4d8c-b229-d27c48afba58 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:35.195652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1229/2428] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:52:35 2022] PUT /identity/v3/projects/4a048153141b485a92cfb09dd6902a91/users/7f551fe9c2d04de184de188eda5a830a/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:52:35.213270 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1200/2429] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:52:35.215038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8580ed0-df72-4b63-9028-9f8af2014071 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:35.360573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8580ed0-df72-4b63-9028-9f8af2014071 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:35.369703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eecd714-2df9-408f-ac8f-9e70e3178b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:35.371335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecd714-2df9-408f-ac8f-9e70e3178b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:35.371839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecd714-2df9-408f-ac8f-9e70e3178b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:35.372330 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecd714-2df9-408f-ac8f-9e70e3178b92 admin admin] PATH_INFO: `/v3/projects/4156a5a77e7e4148b8812fb804257478/users/ab2e292e7ca04edd97df12942b06c53a/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:35.402044 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eecd714-2df9-408f-ac8f-9e70e3178b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4156a5a77e7e4148b8812fb804257478, user_id=ab2e292e7ca04edd97df12942b06c53a, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:35.408875 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eecd714-2df9-408f-ac8f-9e70e3178b92 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:35.453747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1230/2430] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:52:35 2022] PUT /identity/v3/projects/4156a5a77e7e4148b8812fb804257478/users/ab2e292e7ca04edd97df12942b06c53a/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:52:35.484362 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d62e9-eaba-4a1c-bcd6-f07613a12c36 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:35.487870 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d62e9-eaba-4a1c-bcd6-f07613a12c36 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:35.490215 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d62e9-eaba-4a1c-bcd6-f07613a12c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:35.570623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-ce6d62e9-eaba-4a1c-bcd6-f07613a12c36 None None] MFA Rules not processed for user `ab2e292e7ca04edd97df12942b06c53a`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:35.573963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-ce6d62e9-eaba-4a1c-bcd6-f07613a12c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:35.759311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:52:35.879195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1201/2431] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:52:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:52:35.900177 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eecd714-2df9-408f-ac8f-9e70e3178b92 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:35.903434 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8eecd714-2df9-408f-ac8f-9e70e3178b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:36.038417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eecd714-2df9-408f-ac8f-9e70e3178b92 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:36.043535 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832e723-602c-4a0a-a246-47f22f02fe49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:36.044228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832e723-602c-4a0a-a246-47f22f02fe49 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:36.044515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832e723-602c-4a0a-a246-47f22f02fe49 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:36.044795 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832e723-602c-4a0a-a246-47f22f02fe49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:36.071501 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1832e723-602c-4a0a-a246-47f22f02fe49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:36.073151 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1832e723-602c-4a0a-a246-47f22f02fe49 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:36.096993 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1231/2432] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:52:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:52:36.108340 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda3892-5cb7-4ee8-ae48-92a6597c1dc0 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:36.108812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda3892-5cb7-4ee8-ae48-92a6597c1dc0 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:36.109145 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda3892-5cb7-4ee8-ae48-92a6597c1dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:36.137360 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-4dda3892-5cb7-4ee8-ae48-92a6597c1dc0 None None] MFA Rules not processed for user `1a7a81da8acf40849a9ea86e2542ae6a`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:36.138730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4dda3892-5cb7-4ee8-ae48-92a6597c1dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:36.227602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1202/2433] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:52:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:52:44.333541 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80e3c0-d120-41e2-8efe-a28c0776d550 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:44.334139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80e3c0-d120-41e2-8efe-a28c0776d550 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:44.334554 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80e3c0-d120-41e2-8efe-a28c0776d550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:44.400167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-fc80e3c0-d120-41e2-8efe-a28c0776d550 None None] MFA Rules not processed for user `6655366e7eb84a26aa3f7fbe88af62b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:44.402646 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-fc80e3c0-d120-41e2-8efe-a28c0776d550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:44.558266 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:52:44.653189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1232/2434] 10.222.0.20 () {58 vars in 967 bytes} [Mon Jun 27 15:52:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:52:44.683671 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda3892-5cb7-4ee8-ae48-92a6597c1dc0 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:44.685271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4dda3892-5cb7-4ee8-ae48-92a6597c1dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:44.975557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda3892-5cb7-4ee8-ae48-92a6597c1dc0 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:45.000529 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35796b4-4d8f-4154-b5fc-38766efda2cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e83ea69ded040448fe75caa221ad377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:45.002280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35796b4-4d8f-4154-b5fc-38766efda2cb service swift] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:45.003001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35796b4-4d8f-4154-b5fc-38766efda2cb service swift] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:45.003685 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35796b4-4d8f-4154-b5fc-38766efda2cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:45.038466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35796b4-4d8f-4154-b5fc-38766efda2cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:45.055220 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35796b4-4d8f-4154-b5fc-38766efda2cb service swift] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:45.096552 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1203/2435] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jun 27 15:52:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:52:45.596198 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56b449-52b6-49bd-b1c4-7b112805dd19 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:45.596713 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56b449-52b6-49bd-b1c4-7b112805dd19 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:45.596797 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56b449-52b6-49bd-b1c4-7b112805dd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:45.666248 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-ac56b449-52b6-49bd-b1c4-7b112805dd19 None None] MFA Rules not processed for user `7f551fe9c2d04de184de188eda5a830a`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:45.668086 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-ac56b449-52b6-49bd-b1c4-7b112805dd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:45.832482 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1233/2436] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:52:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:52:45.876578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35796b4-4d8f-4154-b5fc-38766efda2cb service swift] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:45.878818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a35796b4-4d8f-4154-b5fc-38766efda2cb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:45.988943 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35796b4-4d8f-4154-b5fc-38766efda2cb service swift] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:45.993674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf71681-0e70-492c-adc0-02af35571724 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493fcde4a45944efbc0b52c44e61b411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:45.994381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf71681-0e70-492c-adc0-02af35571724 service glance] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:45.994669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf71681-0e70-492c-adc0-02af35571724 service glance] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:45.994946 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf71681-0e70-492c-adc0-02af35571724 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:45.996930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-ecf71681-0e70-492c-adc0-02af35571724 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:46.059318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:52:46.111261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf71681-0e70-492c-adc0-02af35571724 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:46.113123 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf71681-0e70-492c-adc0-02af35571724 service glance] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:46.132946 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1204/2437] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:52:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:52:46.277931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26b3e9-9763-48c1-8f63-5a8e4cd4feb6 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:46.278562 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26b3e9-9763-48c1-8f63-5a8e4cd4feb6 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:46.279004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26b3e9-9763-48c1-8f63-5a8e4cd4feb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:46.321890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-7e26b3e9-9763-48c1-8f63-5a8e4cd4feb6 None None] MFA Rules not processed for user `6655366e7eb84a26aa3f7fbe88af62b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:46.323372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-7e26b3e9-9763-48c1-8f63-5a8e4cd4feb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:46.332133 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1234/2438] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jun 27 15:52:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:52:46.345535 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf71681-0e70-492c-adc0-02af35571724 service glance] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:46.359804 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf71681-0e70-492c-adc0-02af35571724 service glance] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:46.365611 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5656e-77e1-4707-aa96-7a8324c0df92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e83ea69ded040448fe75caa221ad377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:46.366323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5656e-77e1-4707-aa96-7a8324c0df92 service swift] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:46.366614 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5656e-77e1-4707-aa96-7a8324c0df92 service swift] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:46.366898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5656e-77e1-4707-aa96-7a8324c0df92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:46.377991 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5656e-77e1-4707-aa96-7a8324c0df92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:46.379831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5656e-77e1-4707-aa96-7a8324c0df92 service swift] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:46.392324 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1205/2439] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jun 27 15:52:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:52:50.629551 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26b3e9-9763-48c1-8f63-5a8e4cd4feb6 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:50.870519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26b3e9-9763-48c1-8f63-5a8e4cd4feb6 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:50.894719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc2706-c450-4808-951c-303c835c0a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:50.896725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc2706-c450-4808-951c-303c835c0a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:50.897534 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc2706-c450-4808-951c-303c835c0a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:50.898747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc2706-c450-4808-951c-303c835c0a79 admin admin] PATH_INFO: `/v3/users/ab2e292e7ca04edd97df12942b06c53a` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:50.902698 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc2706-c450-4808-951c-303c835c0a79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab2e292e7ca04edd97df12942b06c53a)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:50.913957 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc2706-c450-4808-951c-303c835c0a79 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:51.074306 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4fbc2706-c450-4808-951c-303c835c0a79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab2e292e7ca04edd97df12942b06c53a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:51.091446 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:52:51.099819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4fbc2706-c450-4808-951c-303c835c0a79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab2e292e7ca04edd97df12942b06c53a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:51.101605 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4fbc2706-c450-4808-951c-303c835c0a79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab2e292e7ca04edd97df12942b06c53a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:51.115396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4fbc2706-c450-4808-951c-303c835c0a79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab2e292e7ca04edd97df12942b06c53a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:51.130422 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4fbc2706-c450-4808-951c-303c835c0a79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab2e292e7ca04edd97df12942b06c53a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:51.143455 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1235/2440] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:52:50 2022] DELETE /identity/v3/users/ab2e292e7ca04edd97df12942b06c53a => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:52:51.145216 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f8ab9-2d06-454a-b326-465ad531382c None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:51.146257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f8ab9-2d06-454a-b326-465ad531382c None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:51.146734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f8ab9-2d06-454a-b326-465ad531382c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:51.178092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-0a2f8ab9-2d06-454a-b326-465ad531382c None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:51.179386 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-0a2f8ab9-2d06-454a-b326-465ad531382c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:51.335064 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1206/2441] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:52:51.344662 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc2706-c450-4808-951c-303c835c0a79 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:51.346967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-4fbc2706-c450-4808-951c-303c835c0a79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:51.468153 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc2706-c450-4808-951c-303c835c0a79 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:51.473619 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3bde9-aeed-4d19-9968-14afd3ccc88e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:51.474527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3bde9-aeed-4d19-9968-14afd3ccc88e service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:51.474946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3bde9-aeed-4d19-9968-14afd3ccc88e service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:51.475325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3bde9-aeed-4d19-9968-14afd3ccc88e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:51.493990 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3bde9-aeed-4d19-9968-14afd3ccc88e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:51.495895 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3bde9-aeed-4d19-9968-14afd3ccc88e service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:51.509268 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1236/2442] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:52:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:52:51.973041 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f8ab9-2d06-454a-b326-465ad531382c None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:51.976055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-0a2f8ab9-2d06-454a-b326-465ad531382c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:52.038279 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f8ab9-2d06-454a-b326-465ad531382c None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:52.047813 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f3c99-dfc0-4e5f-8305-0ab884581ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:52.048887 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f3c99-dfc0-4e5f-8305-0ab884581ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:52.049291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f3c99-dfc0-4e5f-8305-0ab884581ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:52.049745 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f3c99-dfc0-4e5f-8305-0ab884581ad1 admin admin] PATH_INFO: `/v3/projects/4156a5a77e7e4148b8812fb804257478` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:52.052567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4f3c99-dfc0-4e5f-8305-0ab884581ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4156a5a77e7e4148b8812fb804257478)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:52.059665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4f3c99-dfc0-4e5f-8305-0ab884581ad1 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:52.151382 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-7c4f3c99-dfc0-4e5f-8305-0ab884581ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4156a5a77e7e4148b8812fb804257478', 'request_id': 'req-7c4f3c99-dfc0-4e5f-8305-0ab884581ad1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:52.168070 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-7c4f3c99-dfc0-4e5f-8305-0ab884581ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4156a5a77e7e4148b8812fb804257478', 'request_id': 'req-7c4f3c99-dfc0-4e5f-8305-0ab884581ad1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:52.180444 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-7c4f3c99-dfc0-4e5f-8305-0ab884581ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4156a5a77e7e4148b8812fb804257478', 'request_id': 'req-7c4f3c99-dfc0-4e5f-8305-0ab884581ad1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:52.203593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-7c4f3c99-dfc0-4e5f-8305-0ab884581ad1 admin admin] The token cache is being invalidate because project 4156a5a77e7e4148b8812fb804257478 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:52:52.203933 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-7c4f3c99-dfc0-4e5f-8305-0ab884581ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:52.204587 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-7c4f3c99-dfc0-4e5f-8305-0ab884581ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:52:52.206252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1207/2443] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:52:51 2022] DELETE /identity/v3/projects/4156a5a77e7e4148b8812fb804257478 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:52:56.923603 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7164317-e7f6-4004-a913-0be7d11980d7 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:56.924212 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7164317-e7f6-4004-a913-0be7d11980d7 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:56.924212 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7164317-e7f6-4004-a913-0be7d11980d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:56.944247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-c7164317-e7f6-4004-a913-0be7d11980d7 None None] MFA Rules not processed for user `1320328d4bd04dc28c0c828a0f30e641`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:56.945484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-c7164317-e7f6-4004-a913-0be7d11980d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:57.029023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1237/2444] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:52:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:52:57.069839 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f3c99-dfc0-4e5f-8305-0ab884581ad1 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:57.072354 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-7c4f3c99-dfc0-4e5f-8305-0ab884581ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:57.155401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:52:57.162811 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f3c99-dfc0-4e5f-8305-0ab884581ad1 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:57.168101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551f902-f7bd-4f37-9dc9-524e443b86b6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493fcde4a45944efbc0b52c44e61b411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:57.169130 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551f902-f7bd-4f37-9dc9-524e443b86b6 service glance] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:57.170262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551f902-f7bd-4f37-9dc9-524e443b86b6 service glance] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:57.170828 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551f902-f7bd-4f37-9dc9-524e443b86b6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:57.173834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-c551f902-f7bd-4f37-9dc9-524e443b86b6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:57.267075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c551f902-f7bd-4f37-9dc9-524e443b86b6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:57.268735 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c551f902-f7bd-4f37-9dc9-524e443b86b6 service glance] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:57.285983 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1208/2445] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:52:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:52:58.043531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7164317-e7f6-4004-a913-0be7d11980d7 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:58.046256 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-c7164317-e7f6-4004-a913-0be7d11980d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:58.154873 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7164317-e7f6-4004-a913-0be7d11980d7 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:52:58.159273 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9c101-94c2-4ab5-93d8-8ef0e7c99cae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:52:58.160095 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9c101-94c2-4ab5-93d8-8ef0e7c99cae service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:58.160515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9c101-94c2-4ab5-93d8-8ef0e7c99cae service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:58.160914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9c101-94c2-4ab5-93d8-8ef0e7c99cae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:58.163396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-aea9c101-94c2-4ab5-93d8-8ef0e7c99cae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:58.183959 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea9c101-94c2-4ab5-93d8-8ef0e7c99cae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:52:58.187381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea9c101-94c2-4ab5-93d8-8ef0e7c99cae service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:52:58.208664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1238/2446] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:52:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:52:59.797007 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0958b-b368-4d2e-8eb5-90e13a920a6b None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:52:59.797739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0958b-b368-4d2e-8eb5-90e13a920a6b None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:52:59.797739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0958b-b368-4d2e-8eb5-90e13a920a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:52:59.818332 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-71c0958b-b368-4d2e-8eb5-90e13a920a6b None None] MFA Rules not processed for user `6655366e7eb84a26aa3f7fbe88af62b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:52:59.820150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-71c0958b-b368-4d2e-8eb5-90e13a920a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:59.907523 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1209/2447] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jun 27 15:52:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:52:59.922417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9c101-94c2-4ab5-93d8-8ef0e7c99cae service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:52:59.924129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-aea9c101-94c2-4ab5-93d8-8ef0e7c99cae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:52:59.981505 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:53:00.055501 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9c101-94c2-4ab5-93d8-8ef0e7c99cae service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:00.064673 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c87ccb-499f-4f60-8a11-eb5e58efde00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e83ea69ded040448fe75caa221ad377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:00.065786 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c87ccb-499f-4f60-8a11-eb5e58efde00 service swift] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:00.066292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c87ccb-499f-4f60-8a11-eb5e58efde00 service swift] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:00.067054 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c87ccb-499f-4f60-8a11-eb5e58efde00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:00.079854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c87ccb-499f-4f60-8a11-eb5e58efde00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:00.083046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c87ccb-499f-4f60-8a11-eb5e58efde00 service swift] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:00.100440 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1239/2448] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jun 27 15:52:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 15:53:09.899160 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=58 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:53:10.093486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=19 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:53:10.510919 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0958b-b368-4d2e-8eb5-90e13a920a6b None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:10.512504 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-71c0958b-b368-4d2e-8eb5-90e13a920a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:10.592250 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0958b-b368-4d2e-8eb5-90e13a920a6b None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:10.598759 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b3b61-6091-49be-97e4-fb1af0b4a83d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:10.600238 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b3b61-6091-49be-97e4-fb1af0b4a83d service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:10.600838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b3b61-6091-49be-97e4-fb1af0b4a83d service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:10.601574 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b3b61-6091-49be-97e4-fb1af0b4a83d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:10.604293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a91b3b61-6091-49be-97e4-fb1af0b4a83d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:10.710812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b3b61-6091-49be-97e4-fb1af0b4a83d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:10.713896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b3b61-6091-49be-97e4-fb1af0b4a83d service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:10.732558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1210/2449] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:53:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:53:17.374870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c87ccb-499f-4f60-8a11-eb5e58efde00 service swift] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:17.377001 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f3c87ccb-499f-4f60-8a11-eb5e58efde00 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:17.392185 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c87ccb-499f-4f60-8a11-eb5e58efde00 service swift] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:17.396348 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae62604-a93b-4a2e-af3c-b0849ffd2fb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:17.397029 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae62604-a93b-4a2e-af3c-b0849ffd2fb6 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:17.397316 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae62604-a93b-4a2e-af3c-b0849ffd2fb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:17.397592 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae62604-a93b-4a2e-af3c-b0849ffd2fb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:17.400409 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1ae62604-a93b-4a2e-af3c-b0849ffd2fb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:17.416939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae62604-a93b-4a2e-af3c-b0849ffd2fb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:17.417666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae62604-a93b-4a2e-af3c-b0849ffd2fb6 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:17.430447 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1240/2450] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:53:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:53:19.167245 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a243f-7ccb-49be-8b92-f4bc343dcb01 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:19.167866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a243f-7ccb-49be-8b92-f4bc343dcb01 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:19.168158 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a243f-7ccb-49be-8b92-f4bc343dcb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:19.196168 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-f50a243f-7ccb-49be-8b92-f4bc343dcb01 None None] MFA Rules not processed for user `d35c652539f443d4895b5f81461833ba`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:53:19.197442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f50a243f-7ccb-49be-8b92-f4bc343dcb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:19.207064 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1211/2451] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:53:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:53:19.641056 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae62604-a93b-4a2e-af3c-b0849ffd2fb6 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:19.655583 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae62604-a93b-4a2e-af3c-b0849ffd2fb6 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:19.660555 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5bae3-1312-45ee-a9e4-5c5fa5b98998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:19.661733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5bae3-1312-45ee-a9e4-5c5fa5b98998 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:19.662239 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5bae3-1312-45ee-a9e4-5c5fa5b98998 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:19.662678 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5bae3-1312-45ee-a9e4-5c5fa5b98998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:19.665186 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-79c5bae3-1312-45ee-a9e4-5c5fa5b98998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:19.678531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5bae3-1312-45ee-a9e4-5c5fa5b98998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:19.679999 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5bae3-1312-45ee-a9e4-5c5fa5b98998 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:19.693775 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1241/2452] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:53:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:53:20.159680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a243f-7ccb-49be-8b92-f4bc343dcb01 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:20.162091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f50a243f-7ccb-49be-8b92-f4bc343dcb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:20.290984 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a243f-7ccb-49be-8b92-f4bc343dcb01 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:20.300279 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177294b-c106-4dac-970c-e81094274b64 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffafb68f1bb941588a5217ad83ad907c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:20.301469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177294b-c106-4dac-970c-e81094274b64 service placement] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:20.301923 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177294b-c106-4dac-970c-e81094274b64 service placement] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:20.302364 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177294b-c106-4dac-970c-e81094274b64 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:20.305454 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a177294b-c106-4dac-970c-e81094274b64 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:20.327528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177294b-c106-4dac-970c-e81094274b64 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:20.330429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177294b-c106-4dac-970c-e81094274b64 service placement] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:20.349428 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1212/2453] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:53:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:53:21.236503 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5bae3-1312-45ee-a9e4-5c5fa5b98998 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:21.252062 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5bae3-1312-45ee-a9e4-5c5fa5b98998 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:21.258987 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08390e1-1930-4b90-b084-148568a64a21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:21.260136 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08390e1-1930-4b90-b084-148568a64a21 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:21.260648 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08390e1-1930-4b90-b084-148568a64a21 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:21.261161 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08390e1-1930-4b90-b084-148568a64a21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:21.265171 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f08390e1-1930-4b90-b084-148568a64a21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:21.375961 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08390e1-1930-4b90-b084-148568a64a21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:21.377517 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08390e1-1930-4b90-b084-148568a64a21 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:21.390720 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1242/2454] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:53:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:53:23.814066 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177294b-c106-4dac-970c-e81094274b64 service placement] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:23.817905 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a177294b-c106-4dac-970c-e81094274b64 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:23.886744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177294b-c106-4dac-970c-e81094274b64 service placement] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:23.894667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a581d-7f3e-45c4-8ef4-f3eeb2238fb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:23.895863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a581d-7f3e-45c4-8ef4-f3eeb2238fb9 service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:23.896369 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a581d-7f3e-45c4-8ef4-f3eeb2238fb9 service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:23.896888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a581d-7f3e-45c4-8ef4-f3eeb2238fb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:23.900376 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3a9a581d-7f3e-45c4-8ef4-f3eeb2238fb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:23.914980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a581d-7f3e-45c4-8ef4-f3eeb2238fb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:23.916744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a581d-7f3e-45c4-8ef4-f3eeb2238fb9 service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:23.931335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1213/2455] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:53:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:53:31.383394 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=40 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:53:31.610422 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08390e1-1930-4b90-b084-148568a64a21 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:31.638051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08390e1-1930-4b90-b084-148568a64a21 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:31.643163 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76059670-6f19-4430-9dfc-308b21d6c083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:31.643918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76059670-6f19-4430-9dfc-308b21d6c083 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:31.644410 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76059670-6f19-4430-9dfc-308b21d6c083 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:31.644764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76059670-6f19-4430-9dfc-308b21d6c083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:31.647217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-76059670-6f19-4430-9dfc-308b21d6c083 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:31.771057 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76059670-6f19-4430-9dfc-308b21d6c083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:31.772710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76059670-6f19-4430-9dfc-308b21d6c083 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:31.786116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1243/2456] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:53:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:53:33.926366 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:53:39.115870 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a581d-7f3e-45c4-8ef4-f3eeb2238fb9 service nova] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:39.126429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3a9a581d-7f3e-45c4-8ef4-f3eeb2238fb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:39.181430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a581d-7f3e-45c4-8ef4-f3eeb2238fb9 service nova] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:39.198732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2566feb-9624-4589-a62d-f922f1e63a8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:39.204202 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2566feb-9624-4589-a62d-f922f1e63a8b service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:39.205091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2566feb-9624-4589-a62d-f922f1e63a8b service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:39.205878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2566feb-9624-4589-a62d-f922f1e63a8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:39.211037 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-b2566feb-9624-4589-a62d-f922f1e63a8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:39.298189 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2566feb-9624-4589-a62d-f922f1e63a8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:39.301376 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2566feb-9624-4589-a62d-f922f1e63a8b service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:39.372792 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1214/2457] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:53:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:53:41.781203 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=30 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:53:43.431790 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11ec97-c18a-42b9-b1ae-deb20c046bdc None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:43.432660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11ec97-c18a-42b9-b1ae-deb20c046bdc None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:43.432660 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11ec97-c18a-42b9-b1ae-deb20c046bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:43.453522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-8c11ec97-c18a-42b9-b1ae-deb20c046bdc None None] MFA Rules not processed for user `6890639a0a2e4ca9b103919fdbbffc5b`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:53:43.454862 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8c11ec97-c18a-42b9-b1ae-deb20c046bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:43.541623 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1244/2458] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:53:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:53:43.930797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2566feb-9624-4589-a62d-f922f1e63a8b service nova] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:43.943922 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2566feb-9624-4589-a62d-f922f1e63a8b service nova] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:43.948658 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2441791-3762-4009-b554-0b36f0caae4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:43.950187 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2441791-3762-4009-b554-0b36f0caae4a service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:43.950683 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2441791-3762-4009-b554-0b36f0caae4a service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:43.950931 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2441791-3762-4009-b554-0b36f0caae4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:43.953281 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-b2441791-3762-4009-b554-0b36f0caae4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:43.966403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2441791-3762-4009-b554-0b36f0caae4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:43.968597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2441791-3762-4009-b554-0b36f0caae4a service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:43.980025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1215/2459] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:53:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:53:50.585930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11ec97-c18a-42b9-b1ae-deb20c046bdc None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:50.602314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11ec97-c18a-42b9-b1ae-deb20c046bdc None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:50.610554 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e2a98-31da-44f5-ac0d-77462584c87a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:50.611693 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e2a98-31da-44f5-ac0d-77462584c87a service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:50.612218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e2a98-31da-44f5-ac0d-77462584c87a service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:50.613391 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e2a98-31da-44f5-ac0d-77462584c87a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:50.616694 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e98e2a98-31da-44f5-ac0d-77462584c87a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:50.872443 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e2a98-31da-44f5-ac0d-77462584c87a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:50.876340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e2a98-31da-44f5-ac0d-77462584c87a service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:50.908503 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1245/2460] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:53:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:53:51.318053 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4b32b-0ce8-4173-a7f9-0c28a9f27aed None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:51.318900 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4b32b-0ce8-4173-a7f9-0c28a9f27aed None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:51.319144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4b32b-0ce8-4173-a7f9-0c28a9f27aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:51.350171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-b5d4b32b-0ce8-4173-a7f9-0c28a9f27aed None None] MFA Rules not processed for user `25024a505319480ab00f4c7e58de600c`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:53:51.352815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-b5d4b32b-0ce8-4173-a7f9-0c28a9f27aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:51.450258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1216/2461] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:53:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:53:57.998562 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e2a98-31da-44f5-ac0d-77462584c87a service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:58.033032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4b32b-0ce8-4173-a7f9-0c28a9f27aed None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:58.037461 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e2a98-31da-44f5-ac0d-77462584c87a service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:58.050434 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe8dba-1e37-4fb3-9e31-6472f3de5fe6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffafb68f1bb941588a5217ad83ad907c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:58.051980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe8dba-1e37-4fb3-9e31-6472f3de5fe6 service placement] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:58.054360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe8dba-1e37-4fb3-9e31-6472f3de5fe6 service placement] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:58.055318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe8dba-1e37-4fb3-9e31-6472f3de5fe6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:58.059372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2dfe8dba-1e37-4fb3-9e31-6472f3de5fe6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:58.061101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4b32b-0ce8-4173-a7f9-0c28a9f27aed None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:58.071168 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6332b34-bb43-4707-9fa2-553a9ba947e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:58.072228 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6332b34-bb43-4707-9fa2-553a9ba947e8 service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:58.072756 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6332b34-bb43-4707-9fa2-553a9ba947e8 service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:58.073247 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6332b34-bb43-4707-9fa2-553a9ba947e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:58.076248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f6332b34-bb43-4707-9fa2-553a9ba947e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:58.093528 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe8dba-1e37-4fb3-9e31-6472f3de5fe6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:58.095960 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6332b34-bb43-4707-9fa2-553a9ba947e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:58.097374 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe8dba-1e37-4fb3-9e31-6472f3de5fe6 service placement] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:58.098297 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6332b34-bb43-4707-9fa2-553a9ba947e8 service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:58.124760 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1246/2462] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:53:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:53:58.126648 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe8dba-1e37-4fb3-9e31-6472f3de5fe6 service placement] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:58.128740 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1217/2463] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:53:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:53:58.130146 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2dfe8dba-1e37-4fb3-9e31-6472f3de5fe6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:58.157845 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe8dba-1e37-4fb3-9e31-6472f3de5fe6 service placement] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:58.173915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f79632-778f-4830-b35a-7607a37d791e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:58.175571 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f79632-778f-4830-b35a-7607a37d791e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:58.176109 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f79632-778f-4830-b35a-7607a37d791e admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:58.178923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f79632-778f-4830-b35a-7607a37d791e admin admin] PATH_INFO: `/v3/users/1a7a81da8acf40849a9ea86e2542ae6a` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:58.181322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f79632-778f-4830-b35a-7607a37d791e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a7a81da8acf40849a9ea86e2542ae6a)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:58.184679 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f79632-778f-4830-b35a-7607a37d791e admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:58.268008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b0f79632-778f-4830-b35a-7607a37d791e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a7a81da8acf40849a9ea86e2542ae6a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:53:58.293025 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b0f79632-778f-4830-b35a-7607a37d791e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a7a81da8acf40849a9ea86e2542ae6a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:53:58.294360 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b0f79632-778f-4830-b35a-7607a37d791e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a7a81da8acf40849a9ea86e2542ae6a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:53:58.307520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b0f79632-778f-4830-b35a-7607a37d791e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a7a81da8acf40849a9ea86e2542ae6a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:53:58.320196 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b0f79632-778f-4830-b35a-7607a37d791e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a7a81da8acf40849a9ea86e2542ae6a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:53:58.322880 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:53:58.336946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1247/2464] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:53:58 2022] DELETE /identity/v3/users/1a7a81da8acf40849a9ea86e2542ae6a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:53:58.872954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6332b34-bb43-4707-9fa2-553a9ba947e8 service nova] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:58.906532 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6332b34-bb43-4707-9fa2-553a9ba947e8 service nova] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:58.996017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a144355-9874-4527-b7ca-8253e933f65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:58.997130 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a144355-9874-4527-b7ca-8253e933f65d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:58.997420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a144355-9874-4527-b7ca-8253e933f65d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:58.997753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a144355-9874-4527-b7ca-8253e933f65d admin admin] PATH_INFO: `/v3/projects/f5e448e724fa479ea693168592686391` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:58.999104 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a144355-9874-4527-b7ca-8253e933f65d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5e448e724fa479ea693168592686391)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:59.001081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a144355-9874-4527-b7ca-8253e933f65d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:59.089006 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4a144355-9874-4527-b7ca-8253e933f65d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5e448e724fa479ea693168592686391', 'request_id': 'req-4a144355-9874-4527-b7ca-8253e933f65d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:53:59.101753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4a144355-9874-4527-b7ca-8253e933f65d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5e448e724fa479ea693168592686391', 'request_id': 'req-4a144355-9874-4527-b7ca-8253e933f65d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:53:59.111269 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4a144355-9874-4527-b7ca-8253e933f65d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5e448e724fa479ea693168592686391', 'request_id': 'req-4a144355-9874-4527-b7ca-8253e933f65d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:53:59.116080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:53:59.141531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4a144355-9874-4527-b7ca-8253e933f65d admin admin] The token cache is being invalidate because project f5e448e724fa479ea693168592686391 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:53:59.142106 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4a144355-9874-4527-b7ca-8253e933f65d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:53:59.143433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4a144355-9874-4527-b7ca-8253e933f65d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:53:59.146962 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1218/2465] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:53:58 2022] DELETE /identity/v3/projects/f5e448e724fa479ea693168592686391 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:53:59.155985 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7532a-e913-4010-9843-849fcbbdece0 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:59.157407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7532a-e913-4010-9843-849fcbbdece0 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:59.157957 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7532a-e913-4010-9843-849fcbbdece0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:59.204428 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-91f7532a-e913-4010-9843-849fcbbdece0 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:53:59.206747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-91f7532a-e913-4010-9843-849fcbbdece0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:59.338113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1248/2466] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:53:59.354749 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92d272-d6ba-4927-af70-16f734031ae9 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:59.356105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92d272-d6ba-4927-af70-16f734031ae9 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:59.356726 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92d272-d6ba-4927-af70-16f734031ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:59.391878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-1a92d272-d6ba-4927-af70-16f734031ae9 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:53:59.394475 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1a92d272-d6ba-4927-af70-16f734031ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:59.410634 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1219/2467] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:53:59.416070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7532a-e913-4010-9843-849fcbbdece0 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:59.431284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7532a-e913-4010-9843-849fcbbdece0 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:59.437408 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfa258-a7f8-48e8-9c8d-7af742bac92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:59.438386 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfa258-a7f8-48e8-9c8d-7af742bac92c admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:59.438809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfa258-a7f8-48e8-9c8d-7af742bac92c admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:59.439312 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfa258-a7f8-48e8-9c8d-7af742bac92c admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:59.440308 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfa258-a7f8-48e8-9c8d-7af742bac92c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:53:59.440828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfa258-a7f8-48e8-9c8d-7af742bac92c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:59.442977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfa258-a7f8-48e8-9c8d-7af742bac92c admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:59.457589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1249/2468] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:53:59.465445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92d272-d6ba-4927-af70-16f734031ae9 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:59.483524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92d272-d6ba-4927-af70-16f734031ae9 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:59.491926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec388a77-d42b-471b-8305-666c923bce96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:59.493072 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec388a77-d42b-471b-8305-666c923bce96 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:59.493562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec388a77-d42b-471b-8305-666c923bce96 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:59.494108 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec388a77-d42b-471b-8305-666c923bce96 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:59.495919 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec388a77-d42b-471b-8305-666c923bce96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:59.500004 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec388a77-d42b-471b-8305-666c923bce96 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:59.547011 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1220/2469] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:53:59 2022] POST /identity/v3/projects => generated 383 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:53:59.551772 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfa258-a7f8-48e8-9c8d-7af742bac92c admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:59.564543 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfa258-a7f8-48e8-9c8d-7af742bac92c admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:59.637519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f57d9-1c6f-419f-9be8-fdb66759051c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:59.638253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f57d9-1c6f-419f-9be8-fdb66759051c admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:59.638562 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f57d9-1c6f-419f-9be8-fdb66759051c admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:59.638830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f57d9-1c6f-419f-9be8-fdb66759051c admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:59.639610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f57d9-1c6f-419f-9be8-fdb66759051c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:59.641672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f57d9-1c6f-419f-9be8-fdb66759051c admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:59.672368 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1250/2470] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:53:59 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:53:59.678810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec388a77-d42b-471b-8305-666c923bce96 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:59.697697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec388a77-d42b-471b-8305-666c923bce96 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:59.706691 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d65b5d-47e6-4595-9f40-9e93bfd49043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:59.707969 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d65b5d-47e6-4595-9f40-9e93bfd49043 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:59.708427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d65b5d-47e6-4595-9f40-9e93bfd49043 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:59.708941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d65b5d-47e6-4595-9f40-9e93bfd49043 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:59.710424 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d65b5d-47e6-4595-9f40-9e93bfd49043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:59.713571 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d65b5d-47e6-4595-9f40-9e93bfd49043 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:59.727799 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1221/2471] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:53:59.732981 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f57d9-1c6f-419f-9be8-fdb66759051c admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:59.744804 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f57d9-1c6f-419f-9be8-fdb66759051c admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:59.754786 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e172579-d13b-40b5-9be4-493e0de283c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:59.755484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e172579-d13b-40b5-9be4-493e0de283c0 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:59.755772 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e172579-d13b-40b5-9be4-493e0de283c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:59.756052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e172579-d13b-40b5-9be4-493e0de283c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:59.756806 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e172579-d13b-40b5-9be4-493e0de283c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:59.758543 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e172579-d13b-40b5-9be4-493e0de283c0 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:59.772200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1251/2472] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:53:59.779221 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d65b5d-47e6-4595-9f40-9e93bfd49043 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:59.790574 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d65b5d-47e6-4595-9f40-9e93bfd49043 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:53:59.796846 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e2e99-25bf-4e58-b31f-16a75b809057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:53:59.798096 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e2e99-25bf-4e58-b31f-16a75b809057 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:59.798354 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e2e99-25bf-4e58-b31f-16a75b809057 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:59.798780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e2e99-25bf-4e58-b31f-16a75b809057 admin admin] PATH_INFO: `/v3/projects/90d9fb4d2e024e6eac7eeee72e63e904/users/ce59e0072f1c4b91a2825e5fecde7e23/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:59.821252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e2e99-25bf-4e58-b31f-16a75b809057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90d9fb4d2e024e6eac7eeee72e63e904, user_id=ce59e0072f1c4b91a2825e5fecde7e23, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:53:59.827262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e2e99-25bf-4e58-b31f-16a75b809057 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:53:59.846588 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1222/2473] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:53:59 2022] PUT /identity/v3/projects/90d9fb4d2e024e6eac7eeee72e63e904/users/ce59e0072f1c4b91a2825e5fecde7e23/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:53:59.852731 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69710e87-daab-4df5-a04e-2d5fe6b57df0 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:53:59.853174 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69710e87-daab-4df5-a04e-2d5fe6b57df0 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:53:59.853928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69710e87-daab-4df5-a04e-2d5fe6b57df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:53:59.877686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-69710e87-daab-4df5-a04e-2d5fe6b57df0 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:53:59.879038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-69710e87-daab-4df5-a04e-2d5fe6b57df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:53:59.960849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1252/2474] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:53:59.974749 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e2e99-25bf-4e58-b31f-16a75b809057 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:53:59.976664 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a27e2e99-25bf-4e58-b31f-16a75b809057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:54:00.049269 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e2e99-25bf-4e58-b31f-16a75b809057 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:00.055620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53083168-3523-40a0-b450-a1eb49ee0d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:00.056795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53083168-3523-40a0-b450-a1eb49ee0d43 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:00.057293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53083168-3523-40a0-b450-a1eb49ee0d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:00.058182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53083168-3523-40a0-b450-a1eb49ee0d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:00.075392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53083168-3523-40a0-b450-a1eb49ee0d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:00.078294 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53083168-3523-40a0-b450-a1eb49ee0d43 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:00.095139 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1223/2475] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:53:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:54:08.428799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa684e5-068f-4b43-ab4f-ed8ba64c223f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:08.429613 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa684e5-068f-4b43-ab4f-ed8ba64c223f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:08.430004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa684e5-068f-4b43-ab4f-ed8ba64c223f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:08.442479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:54:08.475966 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-7fa684e5-068f-4b43-ab4f-ed8ba64c223f None None] MFA Rules not processed for user `ce59e0072f1c4b91a2825e5fecde7e23`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:54:08.477485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-7fa684e5-068f-4b43-ab4f-ed8ba64c223f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:54:08.608212 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1253/2476] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:54:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:54:08.660080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53083168-3523-40a0-b450-a1eb49ee0d43 service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:08.663162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-53083168-3523-40a0-b450-a1eb49ee0d43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:54:08.764778 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53083168-3523-40a0-b450-a1eb49ee0d43 service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:08.773997 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ea3b6-366d-44ce-8867-d4ae45904e4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:08.774712 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ea3b6-366d-44ce-8867-d4ae45904e4d service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:08.775061 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ea3b6-366d-44ce-8867-d4ae45904e4d service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:08.775377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ea3b6-366d-44ce-8867-d4ae45904e4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:08.785949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ea3b6-366d-44ce-8867-d4ae45904e4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:08.787721 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ea3b6-366d-44ce-8867-d4ae45904e4d service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:08.801039 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1224/2477] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:54:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:54:11.108343 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e749127-a4f4-477f-9ce4-fb2fbda0bda8 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:11.109065 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e749127-a4f4-477f-9ce4-fb2fbda0bda8 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:11.109694 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e749127-a4f4-477f-9ce4-fb2fbda0bda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:11.142990 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-1e749127-a4f4-477f-9ce4-fb2fbda0bda8 None None] MFA Rules not processed for user `ce59e0072f1c4b91a2825e5fecde7e23`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:54:11.145537 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-1e749127-a4f4-477f-9ce4-fb2fbda0bda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:54:11.158453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1254/2478] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:54:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:54:18.795898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:54:21.133367 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=33 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:54:45.362220 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ea3b6-366d-44ce-8867-d4ae45904e4d service nova] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:45.380420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ea3b6-366d-44ce-8867-d4ae45904e4d service nova] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:45.388342 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c689709-a514-4a7e-9a16-0bb2d69b8990 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:45.389024 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c689709-a514-4a7e-9a16-0bb2d69b8990 service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:45.389307 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c689709-a514-4a7e-9a16-0bb2d69b8990 service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:45.389584 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c689709-a514-4a7e-9a16-0bb2d69b8990 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:45.391931 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-7c689709-a514-4a7e-9a16-0bb2d69b8990 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:54:45.475026 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c689709-a514-4a7e-9a16-0bb2d69b8990 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:45.476763 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c689709-a514-4a7e-9a16-0bb2d69b8990 service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:45.488947 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1225/2479] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:54:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:54:50.755199 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e749127-a4f4-477f-9ce4-fb2fbda0bda8 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:50.773999 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e749127-a4f4-477f-9ce4-fb2fbda0bda8 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:50.780936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e1277-78fa-4e50-972e-e310af25e800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:50.782206 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e1277-78fa-4e50-972e-e310af25e800 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:50.782711 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e1277-78fa-4e50-972e-e310af25e800 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:50.783252 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e1277-78fa-4e50-972e-e310af25e800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:50.786531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-a98e1277-78fa-4e50-972e-e310af25e800 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:54:50.810139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e1277-78fa-4e50-972e-e310af25e800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:50.813907 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98e1277-78fa-4e50-972e-e310af25e800 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:50.830039 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1255/2480] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:54:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:54:55.175740 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c689709-a514-4a7e-9a16-0bb2d69b8990 service nova] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:55.177516 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-7c689709-a514-4a7e-9a16-0bb2d69b8990 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:54:55.262212 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e1277-78fa-4e50-972e-e310af25e800 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:55.264700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-a98e1277-78fa-4e50-972e-e310af25e800 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:54:55.281930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e1277-78fa-4e50-972e-e310af25e800 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:55.290935 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c689709-a514-4a7e-9a16-0bb2d69b8990 service nova] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:55.292786 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f3d37-c52f-47f7-a5ee-4da07727f002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:55.294012 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f3d37-c52f-47f7-a5ee-4da07727f002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:55.294363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f3d37-c52f-47f7-a5ee-4da07727f002 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:55.294838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f3d37-c52f-47f7-a5ee-4da07727f002 admin admin] PATH_INFO: `/v3/users/d35c652539f443d4895b5f81461833ba` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:55.296534 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f3d37-c52f-47f7-a5ee-4da07727f002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d35c652539f443d4895b5f81461833ba)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:55.298792 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3dad12-488d-4606-870d-a787a9cd9fb5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffafb68f1bb941588a5217ad83ad907c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:55.299034 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f3d37-c52f-47f7-a5ee-4da07727f002 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:55.299709 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3dad12-488d-4606-870d-a787a9cd9fb5 service placement] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:55.300091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3dad12-488d-4606-870d-a787a9cd9fb5 service placement] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:55.300457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3dad12-488d-4606-870d-a787a9cd9fb5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:55.302781 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-8f3dad12-488d-4606-870d-a787a9cd9fb5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:54:55.317964 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3dad12-488d-4606-870d-a787a9cd9fb5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:55.319806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3dad12-488d-4606-870d-a787a9cd9fb5 service placement] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:55.334701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1226/2481] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:54:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:54:55.372085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ca6f3d37-c52f-47f7-a5ee-4da07727f002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd35c652539f443d4895b5f81461833ba'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:54:55.392814 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ca6f3d37-c52f-47f7-a5ee-4da07727f002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd35c652539f443d4895b5f81461833ba'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:54:55.394013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ca6f3d37-c52f-47f7-a5ee-4da07727f002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd35c652539f443d4895b5f81461833ba'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:54:55.405338 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ca6f3d37-c52f-47f7-a5ee-4da07727f002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd35c652539f443d4895b5f81461833ba'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:54:55.413828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ca6f3d37-c52f-47f7-a5ee-4da07727f002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd35c652539f443d4895b5f81461833ba'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:54:55.421936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1256/2482] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:54:55 2022] DELETE /identity/v3/users/d35c652539f443d4895b5f81461833ba => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:54:55.678486 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3dad12-488d-4606-870d-a787a9cd9fb5 service placement] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:55.695697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3dad12-488d-4606-870d-a787a9cd9fb5 service placement] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:55.772825 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdbdde-63aa-4624-bef1-8e42ff695ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:55.773754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdbdde-63aa-4624-bef1-8e42ff695ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:55.774042 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdbdde-63aa-4624-bef1-8e42ff695ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:55.774319 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdbdde-63aa-4624-bef1-8e42ff695ad1 admin admin] PATH_INFO: `/v3/projects/5b5846565bf74c5cac3ff5c3168cfd48` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:55.776538 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdbdde-63aa-4624-bef1-8e42ff695ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b5846565bf74c5cac3ff5c3168cfd48)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:55.778568 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdbdde-63aa-4624-bef1-8e42ff695ad1 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:55.824666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:54:55.850663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4bcdbdde-63aa-4624-bef1-8e42ff695ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b5846565bf74c5cac3ff5c3168cfd48', 'request_id': 'req-4bcdbdde-63aa-4624-bef1-8e42ff695ad1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:54:55.863185 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4bcdbdde-63aa-4624-bef1-8e42ff695ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b5846565bf74c5cac3ff5c3168cfd48', 'request_id': 'req-4bcdbdde-63aa-4624-bef1-8e42ff695ad1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:54:55.871607 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4bcdbdde-63aa-4624-bef1-8e42ff695ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b5846565bf74c5cac3ff5c3168cfd48', 'request_id': 'req-4bcdbdde-63aa-4624-bef1-8e42ff695ad1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:54:55.889831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4bcdbdde-63aa-4624-bef1-8e42ff695ad1 admin admin] The token cache is being invalidate because project 5b5846565bf74c5cac3ff5c3168cfd48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:54:55.890116 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4bcdbdde-63aa-4624-bef1-8e42ff695ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:54:55.890784 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4bcdbdde-63aa-4624-bef1-8e42ff695ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:54:55.892399 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1227/2483] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:54:55 2022] DELETE /identity/v3/projects/5b5846565bf74c5cac3ff5c3168cfd48 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:54:55.897440 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d465cbe-5aaa-4d16-9ae0-d8120c5c6591 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:55.897770 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d465cbe-5aaa-4d16-9ae0-d8120c5c6591 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:55.898052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d465cbe-5aaa-4d16-9ae0-d8120c5c6591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:55.919333 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-3d465cbe-5aaa-4d16-9ae0-d8120c5c6591 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:54:55.920874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-3d465cbe-5aaa-4d16-9ae0-d8120c5c6591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:54:56.004346 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1257/2484] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:54:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:54:56.014698 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625b2a2-fef1-425a-bb12-1f815f03342f None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:56.014982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625b2a2-fef1-425a-bb12-1f815f03342f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:56.015266 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625b2a2-fef1-425a-bb12-1f815f03342f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:56.037909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-4625b2a2-fef1-425a-bb12-1f815f03342f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:54:56.039230 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4625b2a2-fef1-425a-bb12-1f815f03342f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:54:56.048789 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1228/2485] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:54:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:54:56.053451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d465cbe-5aaa-4d16-9ae0-d8120c5c6591 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:56.064382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d465cbe-5aaa-4d16-9ae0-d8120c5c6591 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:56.069423 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313563ad-deff-4b55-bcfd-bd4808506b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:56.070132 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313563ad-deff-4b55-bcfd-bd4808506b3e admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:56.070414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313563ad-deff-4b55-bcfd-bd4808506b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:56.070685 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313563ad-deff-4b55-bcfd-bd4808506b3e admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:56.071979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313563ad-deff-4b55-bcfd-bd4808506b3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:54:56.072318 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313563ad-deff-4b55-bcfd-bd4808506b3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:56.074378 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313563ad-deff-4b55-bcfd-bd4808506b3e admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:56.088041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1258/2486] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:54:56.093951 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4625b2a2-fef1-425a-bb12-1f815f03342f None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:56.106502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4625b2a2-fef1-425a-bb12-1f815f03342f None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:56.111591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d2a00-10b7-40a1-b5ac-248fac39b729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:56.112279 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d2a00-10b7-40a1-b5ac-248fac39b729 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:56.112567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d2a00-10b7-40a1-b5ac-248fac39b729 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:56.112852 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d2a00-10b7-40a1-b5ac-248fac39b729 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:56.113566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d2a00-10b7-40a1-b5ac-248fac39b729 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:56.115784 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d2a00-10b7-40a1-b5ac-248fac39b729 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:56.164065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1229/2487] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:54:56 2022] POST /identity/v3/projects => generated 365 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:54:56.170823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313563ad-deff-4b55-bcfd-bd4808506b3e admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:56.187714 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313563ad-deff-4b55-bcfd-bd4808506b3e admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:56.249184 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:54:56.268003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde8cc4-2d2a-411e-aacb-73cc00d06700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:56.269152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde8cc4-2d2a-411e-aacb-73cc00d06700 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:56.269152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde8cc4-2d2a-411e-aacb-73cc00d06700 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:56.269488 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde8cc4-2d2a-411e-aacb-73cc00d06700 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:56.270019 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde8cc4-2d2a-411e-aacb-73cc00d06700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:56.272090 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde8cc4-2d2a-411e-aacb-73cc00d06700 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:56.297033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1259/2488] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:54:56 2022] POST /identity/v3/users => generated 283 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:54:56.302819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d2a00-10b7-40a1-b5ac-248fac39b729 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:56.319796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d2a00-10b7-40a1-b5ac-248fac39b729 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:56.329849 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bb65f-746e-485f-9707-cba1cc74cbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:56.331018 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bb65f-746e-485f-9707-cba1cc74cbef admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:56.331522 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bb65f-746e-485f-9707-cba1cc74cbef admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:56.332035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bb65f-746e-485f-9707-cba1cc74cbef admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:56.333522 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bb65f-746e-485f-9707-cba1cc74cbef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:56.336786 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bb65f-746e-485f-9707-cba1cc74cbef admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:56.354676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1230/2489] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:54:56.360453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde8cc4-2d2a-411e-aacb-73cc00d06700 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:56.378364 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde8cc4-2d2a-411e-aacb-73cc00d06700 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:56.387134 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0eb2b1-9876-4d76-98ba-4eceea317382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:56.388019 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0eb2b1-9876-4d76-98ba-4eceea317382 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:56.388343 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0eb2b1-9876-4d76-98ba-4eceea317382 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:56.388645 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0eb2b1-9876-4d76-98ba-4eceea317382 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:56.389872 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0eb2b1-9876-4d76-98ba-4eceea317382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:56.392711 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0eb2b1-9876-4d76-98ba-4eceea317382 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:56.407832 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1260/2490] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:54:56.409263 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bb65f-746e-485f-9707-cba1cc74cbef admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:56.421772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bb65f-746e-485f-9707-cba1cc74cbef admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:56.426755 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a38d9e-1c72-40fd-b909-57d289aa8b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:56.427672 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a38d9e-1c72-40fd-b909-57d289aa8b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:56.427951 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a38d9e-1c72-40fd-b909-57d289aa8b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:56.428226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a38d9e-1c72-40fd-b909-57d289aa8b1e admin admin] PATH_INFO: `/v3/projects/f8019db026994ebaa1d8408226899832/users/74535c0b427e419ab27ec7f7d30713a8/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:56.453264 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a38d9e-1c72-40fd-b909-57d289aa8b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8019db026994ebaa1d8408226899832, user_id=74535c0b427e419ab27ec7f7d30713a8, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:56.456005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a38d9e-1c72-40fd-b909-57d289aa8b1e admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:56.471301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1231/2491] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:54:56 2022] PUT /identity/v3/projects/f8019db026994ebaa1d8408226899832/users/74535c0b427e419ab27ec7f7d30713a8/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:54:56.487190 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93340745-7a11-4774-abc5-1f4e2c540561 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:56.487491 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93340745-7a11-4774-abc5-1f4e2c540561 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:56.487757 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93340745-7a11-4774-abc5-1f4e2c540561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:56.520347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-93340745-7a11-4774-abc5-1f4e2c540561 None None] MFA Rules not processed for user `74535c0b427e419ab27ec7f7d30713a8`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:54:56.521591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-93340745-7a11-4774-abc5-1f4e2c540561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:54:56.597151 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1261/2492] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:54:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:54:56.602069 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a38d9e-1c72-40fd-b909-57d289aa8b1e admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:56.684048 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a38d9e-1c72-40fd-b909-57d289aa8b1e admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:56.688850 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2050110-4f32-48ff-89aa-a93e863f6f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:56.689527 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2050110-4f32-48ff-89aa-a93e863f6f42 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:56.689840 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2050110-4f32-48ff-89aa-a93e863f6f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:56.690118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2050110-4f32-48ff-89aa-a93e863f6f42 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:56.690841 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2050110-4f32-48ff-89aa-a93e863f6f42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:56.692804 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2050110-4f32-48ff-89aa-a93e863f6f42 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:56.725881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1232/2493] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:54:56 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:54:56.731396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93340745-7a11-4774-abc5-1f4e2c540561 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:56.743100 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93340745-7a11-4774-abc5-1f4e2c540561 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:56.816024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f6709-5d2d-4644-8a4e-e85fc00c410e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:56.816884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f6709-5d2d-4644-8a4e-e85fc00c410e admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:56.817172 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f6709-5d2d-4644-8a4e-e85fc00c410e admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:56.817451 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f6709-5d2d-4644-8a4e-e85fc00c410e admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:56.818206 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f6709-5d2d-4644-8a4e-e85fc00c410e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:56.824795 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f6709-5d2d-4644-8a4e-e85fc00c410e admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:56.846295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1262/2494] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:54:56 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:54:56.850569 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2050110-4f32-48ff-89aa-a93e863f6f42 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:56.869211 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2050110-4f32-48ff-89aa-a93e863f6f42 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:56.877945 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1fd95-0615-4408-b9da-453fbea3a5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:56.879295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1fd95-0615-4408-b9da-453fbea3a5fd admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:56.879864 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1fd95-0615-4408-b9da-453fbea3a5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:56.880451 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1fd95-0615-4408-b9da-453fbea3a5fd admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:56.882578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1fd95-0615-4408-b9da-453fbea3a5fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:56.885863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1fd95-0615-4408-b9da-453fbea3a5fd admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:56.908698 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1233/2495] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:54:56.913079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f6709-5d2d-4644-8a4e-e85fc00c410e admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:56.925541 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f6709-5d2d-4644-8a4e-e85fc00c410e admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:56.934750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd45ba-3db5-4895-80f7-c326f26a71ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:56.936390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd45ba-3db5-4895-80f7-c326f26a71ac admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:56.937058 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd45ba-3db5-4895-80f7-c326f26a71ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:56.937560 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd45ba-3db5-4895-80f7-c326f26a71ac admin admin] PATH_INFO: `/v3/projects/da479ccb8f5044009e1e85d20b987ce1/users/17f5535458dd4483ad8dccc3cc0c0d97/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:56.977243 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd45ba-3db5-4895-80f7-c326f26a71ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=da479ccb8f5044009e1e85d20b987ce1, user_id=17f5535458dd4483ad8dccc3cc0c0d97, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:56.983579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd45ba-3db5-4895-80f7-c326f26a71ac admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:57.007776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1263/2496] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:54:56 2022] PUT /identity/v3/projects/da479ccb8f5044009e1e85d20b987ce1/users/17f5535458dd4483ad8dccc3cc0c0d97/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:54:57.012852 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1fd95-0615-4408-b9da-453fbea3a5fd admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:57.024359 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1fd95-0615-4408-b9da-453fbea3a5fd admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:57.046256 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:54:57.112941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247904e-33f7-4804-8db4-9846c073b8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:57.113746 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247904e-33f7-4804-8db4-9846c073b8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:57.114379 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247904e-33f7-4804-8db4-9846c073b8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:57.114674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247904e-33f7-4804-8db4-9846c073b8e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:57.115518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247904e-33f7-4804-8db4-9846c073b8e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:57.117201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247904e-33f7-4804-8db4-9846c073b8e1 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:57.128868 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1234/2497] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:54:57.133279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd45ba-3db5-4895-80f7-c326f26a71ac admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:57.144948 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd45ba-3db5-4895-80f7-c326f26a71ac admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:57.152117 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917361b-584b-4886-b38b-675419b250f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:57.153762 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917361b-584b-4886-b38b-675419b250f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:57.154289 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917361b-584b-4886-b38b-675419b250f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:57.154804 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917361b-584b-4886-b38b-675419b250f4 admin admin] PATH_INFO: `/v3/projects/da479ccb8f5044009e1e85d20b987ce1/users/17f5535458dd4483ad8dccc3cc0c0d97/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:57.158678 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917361b-584b-4886-b38b-675419b250f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da479ccb8f5044009e1e85d20b987ce1, user_id=17f5535458dd4483ad8dccc3cc0c0d97, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:57.164359 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917361b-584b-4886-b38b-675419b250f4 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:57.185433 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1264/2498] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:54:57 2022] PUT /identity/v3/projects/da479ccb8f5044009e1e85d20b987ce1/users/17f5535458dd4483ad8dccc3cc0c0d97/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:54:57.202396 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c31155-6b8e-4b53-ae98-42fee4d1751f None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:57.202864 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c31155-6b8e-4b53-ae98-42fee4d1751f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:57.203265 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c31155-6b8e-4b53-ae98-42fee4d1751f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:57.240493 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-16c31155-6b8e-4b53-ae98-42fee4d1751f None None] MFA Rules not processed for user `17f5535458dd4483ad8dccc3cc0c0d97`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:54:57.242574 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-16c31155-6b8e-4b53-ae98-42fee4d1751f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:54:57.420836 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1235/2499] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:54:57 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:54:57.441742 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917361b-584b-4886-b38b-675419b250f4 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:57.450929 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8917361b-584b-4886-b38b-675419b250f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:54:57.488109 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:54:57.599752 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917361b-584b-4886-b38b-675419b250f4 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:57.605496 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f1adf-82bc-4db8-9d32-ca8d1bbdafd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:57.606436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f1adf-82bc-4db8-9d32-ca8d1bbdafd5 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:57.606826 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f1adf-82bc-4db8-9d32-ca8d1bbdafd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:57.607302 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f1adf-82bc-4db8-9d32-ca8d1bbdafd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:57.621931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f1adf-82bc-4db8-9d32-ca8d1bbdafd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:57.623681 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f1adf-82bc-4db8-9d32-ca8d1bbdafd5 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:57.638136 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1265/2500] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:54:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:54:57.897236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c31155-6b8e-4b53-ae98-42fee4d1751f None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:54:57.919475 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c31155-6b8e-4b53-ae98-42fee4d1751f None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:54:57.928435 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e310f-7cc3-40d3-9f8c-305d29955484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:54:57.929595 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e310f-7cc3-40d3-9f8c-305d29955484 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:54:57.930153 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e310f-7cc3-40d3-9f8c-305d29955484 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:54:57.930648 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e310f-7cc3-40d3-9f8c-305d29955484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:54:58.040340 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e310f-7cc3-40d3-9f8c-305d29955484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:54:58.043080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e310f-7cc3-40d3-9f8c-305d29955484 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:54:58.056971 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1236/2501] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:54:57 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:55:03.485334 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f1adf-82bc-4db8-9d32-ca8d1bbdafd5 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:03.586073 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f1adf-82bc-4db8-9d32-ca8d1bbdafd5 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:03.594526 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6d6ec-dd32-4268-b289-86ce46f36a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:03.595927 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6d6ec-dd32-4268-b289-86ce46f36a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:03.596270 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6d6ec-dd32-4268-b289-86ce46f36a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:03.596728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6d6ec-dd32-4268-b289-86ce46f36a05 admin admin] PATH_INFO: `/v3/users/74535c0b427e419ab27ec7f7d30713a8` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:03.599043 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6d6ec-dd32-4268-b289-86ce46f36a05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74535c0b427e419ab27ec7f7d30713a8)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:03.603583 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6d6ec-dd32-4268-b289-86ce46f36a05 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:03.677229 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-f7c6d6ec-dd32-4268-b289-86ce46f36a05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74535c0b427e419ab27ec7f7d30713a8'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:03.693549 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-f7c6d6ec-dd32-4268-b289-86ce46f36a05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74535c0b427e419ab27ec7f7d30713a8'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:03.694565 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-f7c6d6ec-dd32-4268-b289-86ce46f36a05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74535c0b427e419ab27ec7f7d30713a8'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:03.706177 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-f7c6d6ec-dd32-4268-b289-86ce46f36a05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74535c0b427e419ab27ec7f7d30713a8'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:03.722478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-f7c6d6ec-dd32-4268-b289-86ce46f36a05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74535c0b427e419ab27ec7f7d30713a8'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:03.735657 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1266/2502] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:55:03 2022] DELETE /identity/v3/users/74535c0b427e419ab27ec7f7d30713a8 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:55:03.740661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6823b-eebe-4080-a4c6-1229af5ba761 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:03.741516 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6823b-eebe-4080-a4c6-1229af5ba761 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:03.741723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6823b-eebe-4080-a4c6-1229af5ba761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:03.763626 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-aef6823b-eebe-4080-a4c6-1229af5ba761 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:03.766020 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-aef6823b-eebe-4080-a4c6-1229af5ba761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:03.829848 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:03.900573 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1237/2503] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:03.917286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6d6ec-dd32-4268-b289-86ce46f36a05 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:03.919632 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f7c6d6ec-dd32-4268-b289-86ce46f36a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:03.988665 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:04.047061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6d6ec-dd32-4268-b289-86ce46f36a05 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:04.054154 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c89466-bf4a-49f2-8f88-cf9f7fa26dda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:04.055336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c89466-bf4a-49f2-8f88-cf9f7fa26dda service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:04.056469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c89466-bf4a-49f2-8f88-cf9f7fa26dda service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:04.056995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c89466-bf4a-49f2-8f88-cf9f7fa26dda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:04.075821 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c89466-bf4a-49f2-8f88-cf9f7fa26dda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:04.078457 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c89466-bf4a-49f2-8f88-cf9f7fa26dda service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:04.096838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1267/2504] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:55:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:55:04.498982 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6823b-eebe-4080-a4c6-1229af5ba761 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:04.500666 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-aef6823b-eebe-4080-a4c6-1229af5ba761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:04.519719 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6823b-eebe-4080-a4c6-1229af5ba761 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:04.529258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a966c4-0605-426e-b185-1bd1e18c44c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:04.531281 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a966c4-0605-426e-b185-1bd1e18c44c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:04.531751 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a966c4-0605-426e-b185-1bd1e18c44c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:04.532236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a966c4-0605-426e-b185-1bd1e18c44c7 admin admin] PATH_INFO: `/v3/projects/f8019db026994ebaa1d8408226899832` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:04.534357 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a966c4-0605-426e-b185-1bd1e18c44c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8019db026994ebaa1d8408226899832)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:04.536818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a966c4-0605-426e-b185-1bd1e18c44c7 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:04.617112 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-61a966c4-0605-426e-b185-1bd1e18c44c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8019db026994ebaa1d8408226899832', 'request_id': 'req-61a966c4-0605-426e-b185-1bd1e18c44c7'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:04.624890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-61a966c4-0605-426e-b185-1bd1e18c44c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8019db026994ebaa1d8408226899832', 'request_id': 'req-61a966c4-0605-426e-b185-1bd1e18c44c7'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:04.631369 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-61a966c4-0605-426e-b185-1bd1e18c44c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8019db026994ebaa1d8408226899832', 'request_id': 'req-61a966c4-0605-426e-b185-1bd1e18c44c7'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:04.645938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-61a966c4-0605-426e-b185-1bd1e18c44c7 admin admin] The token cache is being invalidate because project f8019db026994ebaa1d8408226899832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:55:04.646244 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-61a966c4-0605-426e-b185-1bd1e18c44c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:04.646935 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-61a966c4-0605-426e-b185-1bd1e18c44c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:04.648577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1238/2505] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:55:04 2022] DELETE /identity/v3/projects/f8019db026994ebaa1d8408226899832 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:55:04.653574 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c89466-bf4a-49f2-8f88-cf9f7fa26dda service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:04.666056 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c89466-bf4a-49f2-8f88-cf9f7fa26dda service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:04.732370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de815bba-3517-484e-8b2d-3a3e97e20c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:04.733308 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de815bba-3517-484e-8b2d-3a3e97e20c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:04.733592 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de815bba-3517-484e-8b2d-3a3e97e20c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:04.734031 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de815bba-3517-484e-8b2d-3a3e97e20c2e admin admin] PATH_INFO: `/v3/users/17f5535458dd4483ad8dccc3cc0c0d97` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:04.735346 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de815bba-3517-484e-8b2d-3a3e97e20c2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=17f5535458dd4483ad8dccc3cc0c0d97)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:04.737374 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de815bba-3517-484e-8b2d-3a3e97e20c2e admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:04.810793 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-de815bba-3517-484e-8b2d-3a3e97e20c2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17f5535458dd4483ad8dccc3cc0c0d97'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:04.825543 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-de815bba-3517-484e-8b2d-3a3e97e20c2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17f5535458dd4483ad8dccc3cc0c0d97'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:04.827499 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-de815bba-3517-484e-8b2d-3a3e97e20c2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17f5535458dd4483ad8dccc3cc0c0d97'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:04.836572 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-de815bba-3517-484e-8b2d-3a3e97e20c2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17f5535458dd4483ad8dccc3cc0c0d97'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:04.846218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-de815bba-3517-484e-8b2d-3a3e97e20c2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17f5535458dd4483ad8dccc3cc0c0d97'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:04.854684 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1268/2506] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:55:04 2022] DELETE /identity/v3/users/17f5535458dd4483ad8dccc3cc0c0d97 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:55:05.194596 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a966c4-0605-426e-b185-1bd1e18c44c7 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:05.198430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-61a966c4-0605-426e-b185-1bd1e18c44c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:05.219839 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a966c4-0605-426e-b185-1bd1e18c44c7 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:05.346290 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab1d92-3dee-4999-aee6-14f93b9da7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:05.347842 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab1d92-3dee-4999-aee6-14f93b9da7d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:05.348317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab1d92-3dee-4999-aee6-14f93b9da7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:05.348819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab1d92-3dee-4999-aee6-14f93b9da7d8 admin admin] PATH_INFO: `/v3/projects/da479ccb8f5044009e1e85d20b987ce1` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:05.351122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab1d92-3dee-4999-aee6-14f93b9da7d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da479ccb8f5044009e1e85d20b987ce1)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:05.356200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ab1d92-3dee-4999-aee6-14f93b9da7d8 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:05.458193 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-64ab1d92-3dee-4999-aee6-14f93b9da7d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da479ccb8f5044009e1e85d20b987ce1', 'request_id': 'req-64ab1d92-3dee-4999-aee6-14f93b9da7d8'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:05.467623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-64ab1d92-3dee-4999-aee6-14f93b9da7d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da479ccb8f5044009e1e85d20b987ce1', 'request_id': 'req-64ab1d92-3dee-4999-aee6-14f93b9da7d8'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:05.478103 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-64ab1d92-3dee-4999-aee6-14f93b9da7d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da479ccb8f5044009e1e85d20b987ce1', 'request_id': 'req-64ab1d92-3dee-4999-aee6-14f93b9da7d8'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:05.481715 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:05.498788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-64ab1d92-3dee-4999-aee6-14f93b9da7d8 admin admin] The token cache is being invalidate because project da479ccb8f5044009e1e85d20b987ce1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:55:05.499101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-64ab1d92-3dee-4999-aee6-14f93b9da7d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:05.500176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-64ab1d92-3dee-4999-aee6-14f93b9da7d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:05.507422 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1239/2507] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:55:05 2022] DELETE /identity/v3/projects/da479ccb8f5044009e1e85d20b987ce1 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:55:05.510217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce01fc-0df9-45db-b973-5d0f6dd157ab None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:05.510620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce01fc-0df9-45db-b973-5d0f6dd157ab None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:05.510882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce01fc-0df9-45db-b973-5d0f6dd157ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:05.534377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-dbce01fc-0df9-45db-b973-5d0f6dd157ab None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:05.535642 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-dbce01fc-0df9-45db-b973-5d0f6dd157ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:05.595703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:05.622426 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1269/2508] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:55:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:05.631479 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fb8e5-18f0-4fb9-a94e-fcbc9bb409ad None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:05.632010 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fb8e5-18f0-4fb9-a94e-fcbc9bb409ad None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:05.632506 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fb8e5-18f0-4fb9-a94e-fcbc9bb409ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:05.655049 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-0e1fb8e5-18f0-4fb9-a94e-fcbc9bb409ad None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:05.656897 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-0e1fb8e5-18f0-4fb9-a94e-fcbc9bb409ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:05.668029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1240/2509] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:55:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:05.674782 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce01fc-0df9-45db-b973-5d0f6dd157ab None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:05.693397 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce01fc-0df9-45db-b973-5d0f6dd157ab None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:05.702971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac789e61-5c54-4074-a4da-0a300fb4eeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:05.704124 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac789e61-5c54-4074-a4da-0a300fb4eeff admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:05.704619 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac789e61-5c54-4074-a4da-0a300fb4eeff admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:05.705109 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac789e61-5c54-4074-a4da-0a300fb4eeff admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:05.706598 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac789e61-5c54-4074-a4da-0a300fb4eeff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:55:05.707143 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac789e61-5c54-4074-a4da-0a300fb4eeff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:05.710384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac789e61-5c54-4074-a4da-0a300fb4eeff admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:05.733654 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1270/2510] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:55:05.738705 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1fb8e5-18f0-4fb9-a94e-fcbc9bb409ad None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:05.750221 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1fb8e5-18f0-4fb9-a94e-fcbc9bb409ad None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:05.755178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3dcfb9-e68f-45f6-84a3-b7641abe2603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:05.755871 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3dcfb9-e68f-45f6-84a3-b7641abe2603 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:05.756158 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3dcfb9-e68f-45f6-84a3-b7641abe2603 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:05.756438 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3dcfb9-e68f-45f6-84a3-b7641abe2603 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:05.757146 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3dcfb9-e68f-45f6-84a3-b7641abe2603 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:05.759748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3dcfb9-e68f-45f6-84a3-b7641abe2603 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:05.791759 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1241/2511] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:55:05 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:55:05.795826 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac789e61-5c54-4074-a4da-0a300fb4eeff admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:05.816012 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac789e61-5c54-4074-a4da-0a300fb4eeff admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:05.931898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc6101-6cc6-43df-bcae-1c831c3108d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:05.933078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc6101-6cc6-43df-bcae-1c831c3108d6 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:05.933591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc6101-6cc6-43df-bcae-1c831c3108d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:05.937561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc6101-6cc6-43df-bcae-1c831c3108d6 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:05.940517 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc6101-6cc6-43df-bcae-1c831c3108d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:05.944523 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc6101-6cc6-43df-bcae-1c831c3108d6 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:05.986808 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1271/2512] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:55:05 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:55:05.988772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3dcfb9-e68f-45f6-84a3-b7641abe2603 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:06.008436 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3dcfb9-e68f-45f6-84a3-b7641abe2603 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:06.017186 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac82b74-6013-40cb-8b8e-b6cff327de44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:06.018423 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac82b74-6013-40cb-8b8e-b6cff327de44 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:06.018934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac82b74-6013-40cb-8b8e-b6cff327de44 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:06.019439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac82b74-6013-40cb-8b8e-b6cff327de44 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:06.020860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac82b74-6013-40cb-8b8e-b6cff327de44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:06.024195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac82b74-6013-40cb-8b8e-b6cff327de44 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:06.048806 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1242/2513] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:55:06.050708 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc6101-6cc6-43df-bcae-1c831c3108d6 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:06.066913 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc6101-6cc6-43df-bcae-1c831c3108d6 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:06.075890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1b034-6b18-4101-b33f-97295e10861b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:06.077038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1b034-6b18-4101-b33f-97295e10861b admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:06.077544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1b034-6b18-4101-b33f-97295e10861b admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:06.078083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1b034-6b18-4101-b33f-97295e10861b admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:06.079467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1b034-6b18-4101-b33f-97295e10861b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:06.082708 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1b034-6b18-4101-b33f-97295e10861b admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:06.099485 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1272/2514] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:55:06.106741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac82b74-6013-40cb-8b8e-b6cff327de44 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:06.126790 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac82b74-6013-40cb-8b8e-b6cff327de44 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:06.135786 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c4d84-7478-45fd-9342-3c6e9bf0e413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:06.137682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c4d84-7478-45fd-9342-3c6e9bf0e413 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:06.138028 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c4d84-7478-45fd-9342-3c6e9bf0e413 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:06.138389 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c4d84-7478-45fd-9342-3c6e9bf0e413 admin admin] PATH_INFO: `/v3/projects/c3487602aaf94ec4912e713c356e0a5a/users/280ae39584694c418bf6b7a59df7eee4/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:06.158934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c4d84-7478-45fd-9342-3c6e9bf0e413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3487602aaf94ec4912e713c356e0a5a, user_id=280ae39584694c418bf6b7a59df7eee4, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:06.161774 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c4d84-7478-45fd-9342-3c6e9bf0e413 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:06.179372 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1243/2515] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:55:06 2022] PUT /identity/v3/projects/c3487602aaf94ec4912e713c356e0a5a/users/280ae39584694c418bf6b7a59df7eee4/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:55:06.202122 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d52da-9a79-40b4-9782-45786d0c0578 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:06.202639 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d52da-9a79-40b4-9782-45786d0c0578 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:06.203146 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d52da-9a79-40b4-9782-45786d0c0578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:06.255027 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-279d52da-9a79-40b4-9782-45786d0c0578 None None] MFA Rules not processed for user `280ae39584694c418bf6b7a59df7eee4`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:06.257785 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-279d52da-9a79-40b4-9782-45786d0c0578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:06.360135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1273/2516] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:55:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:06.366545 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c4d84-7478-45fd-9342-3c6e9bf0e413 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:06.484305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c4d84-7478-45fd-9342-3c6e9bf0e413 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:06.492855 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6d77f-04cb-430c-94c2-6a3fac5323b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:06.494588 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6d77f-04cb-430c-94c2-6a3fac5323b6 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:06.495335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6d77f-04cb-430c-94c2-6a3fac5323b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:06.495821 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6d77f-04cb-430c-94c2-6a3fac5323b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:06.497086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6d77f-04cb-430c-94c2-6a3fac5323b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:06.500860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6d77f-04cb-430c-94c2-6a3fac5323b6 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:06.592586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1244/2517] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:55:06 2022] POST /identity/v3/projects => generated 387 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:55:06.605875 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d52da-9a79-40b4-9782-45786d0c0578 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:06.635468 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d52da-9a79-40b4-9782-45786d0c0578 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:06.758074 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:06.792664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f3839-057a-4bca-8063-ce7e4b5d5b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:06.793366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f3839-057a-4bca-8063-ce7e4b5d5b5f admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:06.793675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f3839-057a-4bca-8063-ce7e4b5d5b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:06.793968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f3839-057a-4bca-8063-ce7e4b5d5b5f admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:06.794703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f3839-057a-4bca-8063-ce7e4b5d5b5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:06.796968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f3839-057a-4bca-8063-ce7e4b5d5b5f admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:06.847885 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1274/2518] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:55:06 2022] POST /identity/v3/users => generated 300 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:55:06.856474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6d77f-04cb-430c-94c2-6a3fac5323b6 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:06.887052 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6d77f-04cb-430c-94c2-6a3fac5323b6 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:06.894160 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ab538-fe94-489a-9654-a641ee8aaafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:06.894924 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ab538-fe94-489a-9654-a641ee8aaafd admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:06.895353 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ab538-fe94-489a-9654-a641ee8aaafd admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:06.895601 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ab538-fe94-489a-9654-a641ee8aaafd admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:06.896264 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ab538-fe94-489a-9654-a641ee8aaafd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:06.897964 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ab538-fe94-489a-9654-a641ee8aaafd admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:06.911788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1245/2519] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:55:06.916628 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f3839-057a-4bca-8063-ce7e4b5d5b5f admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:06.931189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f3839-057a-4bca-8063-ce7e4b5d5b5f admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:06.940800 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae7749-97ef-4eed-b914-55b228204d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:06.942922 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae7749-97ef-4eed-b914-55b228204d8a admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:06.943268 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae7749-97ef-4eed-b914-55b228204d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:06.943569 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae7749-97ef-4eed-b914-55b228204d8a admin admin] PATH_INFO: `/v3/projects/639dbceae2b84fe688cae42fe32e6c6e/users/1258a353c2444f40a6de01fc2b5f8a99/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:06.964176 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae7749-97ef-4eed-b914-55b228204d8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=639dbceae2b84fe688cae42fe32e6c6e, user_id=1258a353c2444f40a6de01fc2b5f8a99, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:06.966983 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae7749-97ef-4eed-b914-55b228204d8a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:06.983436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1275/2520] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:55:06 2022] PUT /identity/v3/projects/639dbceae2b84fe688cae42fe32e6c6e/users/1258a353c2444f40a6de01fc2b5f8a99/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:55:06.987697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ab538-fe94-489a-9654-a641ee8aaafd admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:06.999864 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ab538-fe94-489a-9654-a641ee8aaafd admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:07.072579 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73efb2-ce51-4a1a-9e26-b7d54da48c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:07.073831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73efb2-ce51-4a1a-9e26-b7d54da48c01 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:07.074280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73efb2-ce51-4a1a-9e26-b7d54da48c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:07.074687 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73efb2-ce51-4a1a-9e26-b7d54da48c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:07.075864 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73efb2-ce51-4a1a-9e26-b7d54da48c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:07.078759 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73efb2-ce51-4a1a-9e26-b7d54da48c01 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:07.087652 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:07.097296 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1246/2521] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:55:07.101616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae7749-97ef-4eed-b914-55b228204d8a admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:07.114825 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae7749-97ef-4eed-b914-55b228204d8a admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:07.121060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc91f22-06d7-488e-993d-4347ac2e8a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:07.122033 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc91f22-06d7-488e-993d-4347ac2e8a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:07.122346 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc91f22-06d7-488e-993d-4347ac2e8a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:07.122621 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc91f22-06d7-488e-993d-4347ac2e8a83 admin admin] PATH_INFO: `/v3/projects/639dbceae2b84fe688cae42fe32e6c6e/users/1258a353c2444f40a6de01fc2b5f8a99/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:07.124827 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc91f22-06d7-488e-993d-4347ac2e8a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=639dbceae2b84fe688cae42fe32e6c6e, user_id=1258a353c2444f40a6de01fc2b5f8a99, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:07.127585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc91f22-06d7-488e-993d-4347ac2e8a83 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:07.141257 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1276/2522] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:55:07 2022] PUT /identity/v3/projects/639dbceae2b84fe688cae42fe32e6c6e/users/1258a353c2444f40a6de01fc2b5f8a99/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:55:07.152904 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536779a0-8ad8-4b7f-ba32-307739a7e71d None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:07.153294 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536779a0-8ad8-4b7f-ba32-307739a7e71d None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:07.153586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536779a0-8ad8-4b7f-ba32-307739a7e71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:07.189958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-536779a0-8ad8-4b7f-ba32-307739a7e71d None None] MFA Rules not processed for user `1258a353c2444f40a6de01fc2b5f8a99`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:07.191235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-536779a0-8ad8-4b7f-ba32-307739a7e71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:07.287946 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1247/2523] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:55:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:55:07.301400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc91f22-06d7-488e-993d-4347ac2e8a83 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:07.303296 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-3bc91f22-06d7-488e-993d-4347ac2e8a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:07.391432 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc91f22-06d7-488e-993d-4347ac2e8a83 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:07.396388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccfdc8-c218-491c-929e-0891330d41cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:07.397193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccfdc8-c218-491c-929e-0891330d41cb service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:07.397882 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccfdc8-c218-491c-929e-0891330d41cb service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:07.398327 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccfdc8-c218-491c-929e-0891330d41cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:07.410806 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccfdc8-c218-491c-929e-0891330d41cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:07.413076 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccfdc8-c218-491c-929e-0891330d41cb service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:07.428559 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1277/2524] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:55:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:55:07.978042 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a21b3-1a11-420d-9feb-a37e6731a85e None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:07.978991 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a21b3-1a11-420d-9feb-a37e6731a85e None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:07.979592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0a21b3-1a11-420d-9feb-a37e6731a85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:08.010276 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-7d0a21b3-1a11-420d-9feb-a37e6731a85e None None] MFA Rules not processed for user `6655366e7eb84a26aa3f7fbe88af62b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:08.011588 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-7d0a21b3-1a11-420d-9feb-a37e6731a85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:08.110609 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1248/2525] 10.222.0.20 () {58 vars in 967 bytes} [Mon Jun 27 15:55:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:08.201443 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccfdc8-c218-491c-929e-0891330d41cb service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:08.203940 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-93ccfdc8-c218-491c-929e-0891330d41cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:08.318628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a21b3-1a11-420d-9feb-a37e6731a85e None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:08.349263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccfdc8-c218-491c-929e-0891330d41cb service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:08.359030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916a59c-c326-4508-9d40-2c256aeb0b4d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e83ea69ded040448fe75caa221ad377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:08.360343 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916a59c-c326-4508-9d40-2c256aeb0b4d service swift] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:08.360868 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916a59c-c326-4508-9d40-2c256aeb0b4d service swift] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:08.361374 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916a59c-c326-4508-9d40-2c256aeb0b4d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:08.379228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916a59c-c326-4508-9d40-2c256aeb0b4d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:08.382654 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916a59c-c326-4508-9d40-2c256aeb0b4d service swift] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:08.395132 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1278/2526] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jun 27 15:55:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:55:08.406343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0a21b3-1a11-420d-9feb-a37e6731a85e None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:08.411429 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751573da-9484-4d59-9455-8a57680b6d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:08.412364 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751573da-9484-4d59-9455-8a57680b6d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:08.412836 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751573da-9484-4d59-9455-8a57680b6d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:08.412974 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751573da-9484-4d59-9455-8a57680b6d3a admin admin] PATH_INFO: `/v3/users/280ae39584694c418bf6b7a59df7eee4` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:08.414370 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751573da-9484-4d59-9455-8a57680b6d3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=280ae39584694c418bf6b7a59df7eee4)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:08.417254 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751573da-9484-4d59-9455-8a57680b6d3a admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:08.471986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:08.496462 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-751573da-9484-4d59-9455-8a57680b6d3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '280ae39584694c418bf6b7a59df7eee4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:08.521703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-751573da-9484-4d59-9455-8a57680b6d3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '280ae39584694c418bf6b7a59df7eee4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:08.535490 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-751573da-9484-4d59-9455-8a57680b6d3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '280ae39584694c418bf6b7a59df7eee4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:08.536331 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-751573da-9484-4d59-9455-8a57680b6d3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '280ae39584694c418bf6b7a59df7eee4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:08.537355 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-751573da-9484-4d59-9455-8a57680b6d3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '280ae39584694c418bf6b7a59df7eee4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:08.559133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1249/2527] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:55:08 2022] DELETE /identity/v3/users/280ae39584694c418bf6b7a59df7eee4 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:55:08.565832 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ce8f7-fa50-4076-b5fb-74e4440946ba None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:08.566343 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ce8f7-fa50-4076-b5fb-74e4440946ba None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:08.566689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ce8f7-fa50-4076-b5fb-74e4440946ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:08.596440 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-050ce8f7-fa50-4076-b5fb-74e4440946ba None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:08.597955 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-050ce8f7-fa50-4076-b5fb-74e4440946ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:08.638929 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:08.698958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1279/2528] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:08.715939 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751573da-9484-4d59-9455-8a57680b6d3a admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:08.718047 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-751573da-9484-4d59-9455-8a57680b6d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:08.795116 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751573da-9484-4d59-9455-8a57680b6d3a admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:08.799851 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfdb0c-4ed9-4267-a059-9f9a7636abfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:08.800547 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfdb0c-4ed9-4267-a059-9f9a7636abfc service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:08.800863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfdb0c-4ed9-4267-a059-9f9a7636abfc service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:08.801150 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfdb0c-4ed9-4267-a059-9f9a7636abfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:08.812958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facfdb0c-4ed9-4267-a059-9f9a7636abfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:08.814468 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facfdb0c-4ed9-4267-a059-9f9a7636abfc service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:08.828286 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1250/2529] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:55:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:55:09.251427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ce8f7-fa50-4076-b5fb-74e4440946ba None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:09.255738 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-050ce8f7-fa50-4076-b5fb-74e4440946ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:09.289483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ce8f7-fa50-4076-b5fb-74e4440946ba None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:09.309719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0644de8-bc34-4771-8d29-3d8cdeb64b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:09.311577 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0644de8-bc34-4771-8d29-3d8cdeb64b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:09.312860 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0644de8-bc34-4771-8d29-3d8cdeb64b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:09.313390 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0644de8-bc34-4771-8d29-3d8cdeb64b02 admin admin] PATH_INFO: `/v3/projects/c3487602aaf94ec4912e713c356e0a5a` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:09.315380 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0644de8-bc34-4771-8d29-3d8cdeb64b02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3487602aaf94ec4912e713c356e0a5a)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:09.317884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0644de8-bc34-4771-8d29-3d8cdeb64b02 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:09.402853 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b0644de8-bc34-4771-8d29-3d8cdeb64b02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3487602aaf94ec4912e713c356e0a5a', 'request_id': 'req-b0644de8-bc34-4771-8d29-3d8cdeb64b02'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:09.411430 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b0644de8-bc34-4771-8d29-3d8cdeb64b02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3487602aaf94ec4912e713c356e0a5a', 'request_id': 'req-b0644de8-bc34-4771-8d29-3d8cdeb64b02'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:09.423829 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b0644de8-bc34-4771-8d29-3d8cdeb64b02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3487602aaf94ec4912e713c356e0a5a', 'request_id': 'req-b0644de8-bc34-4771-8d29-3d8cdeb64b02'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:09.451569 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b0644de8-bc34-4771-8d29-3d8cdeb64b02 admin admin] The token cache is being invalidate because project c3487602aaf94ec4912e713c356e0a5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:55:09.452097 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b0644de8-bc34-4771-8d29-3d8cdeb64b02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:09.453368 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b0644de8-bc34-4771-8d29-3d8cdeb64b02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:09.463641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfdb0c-4ed9-4267-a059-9f9a7636abfc service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:09.471325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1280/2530] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:55:09 2022] DELETE /identity/v3/projects/c3487602aaf94ec4912e713c356e0a5a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:55:09.486835 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfdb0c-4ed9-4267-a059-9f9a7636abfc service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:09.580343 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18137a1-129a-42ff-9800-057546ce940f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:09.581459 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18137a1-129a-42ff-9800-057546ce940f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:09.582543 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18137a1-129a-42ff-9800-057546ce940f admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:09.582996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18137a1-129a-42ff-9800-057546ce940f admin admin] PATH_INFO: `/v3/users/1258a353c2444f40a6de01fc2b5f8a99` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:09.585200 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18137a1-129a-42ff-9800-057546ce940f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1258a353c2444f40a6de01fc2b5f8a99)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:09.588807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18137a1-129a-42ff-9800-057546ce940f admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:09.668006 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b18137a1-129a-42ff-9800-057546ce940f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1258a353c2444f40a6de01fc2b5f8a99'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:09.675540 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:09.688996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b18137a1-129a-42ff-9800-057546ce940f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1258a353c2444f40a6de01fc2b5f8a99'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:09.700098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b18137a1-129a-42ff-9800-057546ce940f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1258a353c2444f40a6de01fc2b5f8a99'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:09.701272 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b18137a1-129a-42ff-9800-057546ce940f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1258a353c2444f40a6de01fc2b5f8a99'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:09.702379 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-b18137a1-129a-42ff-9800-057546ce940f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1258a353c2444f40a6de01fc2b5f8a99'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:09.725761 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1251/2531] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:55:09 2022] DELETE /identity/v3/users/1258a353c2444f40a6de01fc2b5f8a99 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:55:10.247273 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0644de8-bc34-4771-8d29-3d8cdeb64b02 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:10.248814 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-b0644de8-bc34-4771-8d29-3d8cdeb64b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:10.294361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0644de8-bc34-4771-8d29-3d8cdeb64b02 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:10.433143 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c33a42-69e5-482d-a99f-1ee9762a7265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:10.435332 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c33a42-69e5-482d-a99f-1ee9762a7265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:10.438985 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c33a42-69e5-482d-a99f-1ee9762a7265 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:10.440051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c33a42-69e5-482d-a99f-1ee9762a7265 admin admin] PATH_INFO: `/v3/projects/639dbceae2b84fe688cae42fe32e6c6e` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:10.443362 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c33a42-69e5-482d-a99f-1ee9762a7265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=639dbceae2b84fe688cae42fe32e6c6e)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:10.448897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c33a42-69e5-482d-a99f-1ee9762a7265 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:10.537601 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-51c33a42-69e5-482d-a99f-1ee9762a7265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '639dbceae2b84fe688cae42fe32e6c6e', 'request_id': 'req-51c33a42-69e5-482d-a99f-1ee9762a7265'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:10.549065 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-51c33a42-69e5-482d-a99f-1ee9762a7265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '639dbceae2b84fe688cae42fe32e6c6e', 'request_id': 'req-51c33a42-69e5-482d-a99f-1ee9762a7265'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:10.553223 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:10.565687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-51c33a42-69e5-482d-a99f-1ee9762a7265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '639dbceae2b84fe688cae42fe32e6c6e', 'request_id': 'req-51c33a42-69e5-482d-a99f-1ee9762a7265'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:10.596620 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-51c33a42-69e5-482d-a99f-1ee9762a7265 admin admin] The token cache is being invalidate because project 639dbceae2b84fe688cae42fe32e6c6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:55:10.601359 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-51c33a42-69e5-482d-a99f-1ee9762a7265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:10.602596 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-51c33a42-69e5-482d-a99f-1ee9762a7265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:10.605402 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1281/2532] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:55:10 2022] DELETE /identity/v3/projects/639dbceae2b84fe688cae42fe32e6c6e => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:55:10.612003 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c2978-9924-415f-868e-ea04ab6e7b10 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:10.614446 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c2978-9924-415f-868e-ea04ab6e7b10 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:10.614972 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c2978-9924-415f-868e-ea04ab6e7b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:10.642425 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-1f5c2978-9924-415f-868e-ea04ab6e7b10 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:10.644405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1f5c2978-9924-415f-868e-ea04ab6e7b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:10.801879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1252/2533] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:10.812107 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdd3a8-bcd2-4de9-a8aa-e3e5d0165bed None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:10.812677 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdd3a8-bcd2-4de9-a8aa-e3e5d0165bed None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:10.812982 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdd3a8-bcd2-4de9-a8aa-e3e5d0165bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:10.838490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-09fdd3a8-bcd2-4de9-a8aa-e3e5d0165bed None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:10.839970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-09fdd3a8-bcd2-4de9-a8aa-e3e5d0165bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:10.849469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1282/2534] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:55:10.860825 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c2978-9924-415f-868e-ea04ab6e7b10 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:10.875177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c2978-9924-415f-868e-ea04ab6e7b10 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:10.881198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ed02c-fd16-4ff4-9aae-e04c0fcfc21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:10.881886 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ed02c-fd16-4ff4-9aae-e04c0fcfc21f admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:10.882245 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ed02c-fd16-4ff4-9aae-e04c0fcfc21f admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:10.882592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ed02c-fd16-4ff4-9aae-e04c0fcfc21f admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:10.884059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ed02c-fd16-4ff4-9aae-e04c0fcfc21f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:55:10.884489 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ed02c-fd16-4ff4-9aae-e04c0fcfc21f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:10.886619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ed02c-fd16-4ff4-9aae-e04c0fcfc21f admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:10.908462 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fdd3a8-bcd2-4de9-a8aa-e3e5d0165bed None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:10.908773 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1253/2535] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:55:10.925697 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fdd3a8-bcd2-4de9-a8aa-e3e5d0165bed None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:10.941626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815842e-b482-4196-8409-265fc408cfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:10.942829 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815842e-b482-4196-8409-265fc408cfb8 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:10.943346 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815842e-b482-4196-8409-265fc408cfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:10.943863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815842e-b482-4196-8409-265fc408cfb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:10.945216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815842e-b482-4196-8409-265fc408cfb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:10.955436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815842e-b482-4196-8409-265fc408cfb8 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:11.033945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1283/2536] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:55:10 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:55:11.039026 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ed02c-fd16-4ff4-9aae-e04c0fcfc21f admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:11.052949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ed02c-fd16-4ff4-9aae-e04c0fcfc21f admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:11.168502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0171e-c54d-48dd-bf2a-eff6d5447962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:11.169432 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0171e-c54d-48dd-bf2a-eff6d5447962 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:11.170029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0171e-c54d-48dd-bf2a-eff6d5447962 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:11.170759 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0171e-c54d-48dd-bf2a-eff6d5447962 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:11.172007 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0171e-c54d-48dd-bf2a-eff6d5447962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:11.174856 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0171e-c54d-48dd-bf2a-eff6d5447962 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:11.218221 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1254/2537] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:55:11 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:55:11.223108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815842e-b482-4196-8409-265fc408cfb8 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:11.239954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815842e-b482-4196-8409-265fc408cfb8 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:11.248417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bce4d9-28c1-484c-b1f9-bfcc1dd5de4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:11.249405 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bce4d9-28c1-484c-b1f9-bfcc1dd5de4d admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:11.249864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bce4d9-28c1-484c-b1f9-bfcc1dd5de4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:11.250272 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bce4d9-28c1-484c-b1f9-bfcc1dd5de4d admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:11.251165 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bce4d9-28c1-484c-b1f9-bfcc1dd5de4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:11.253086 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bce4d9-28c1-484c-b1f9-bfcc1dd5de4d admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:11.268357 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1284/2538] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:55:11.272491 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0171e-c54d-48dd-bf2a-eff6d5447962 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:11.284811 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0171e-c54d-48dd-bf2a-eff6d5447962 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:11.292625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c349139-c5ce-411e-8f45-99e5b6b59e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:11.294189 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c349139-c5ce-411e-8f45-99e5b6b59e3d admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:11.294718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c349139-c5ce-411e-8f45-99e5b6b59e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:11.295229 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c349139-c5ce-411e-8f45-99e5b6b59e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:11.296441 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c349139-c5ce-411e-8f45-99e5b6b59e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:11.298788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c349139-c5ce-411e-8f45-99e5b6b59e3d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:11.323407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1255/2539] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:55:11.327794 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bce4d9-28c1-484c-b1f9-bfcc1dd5de4d admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:11.339335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bce4d9-28c1-484c-b1f9-bfcc1dd5de4d admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:11.344638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405fe88-3802-44cf-aacd-93be21a5a821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:11.345479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405fe88-3802-44cf-aacd-93be21a5a821 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:11.345861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405fe88-3802-44cf-aacd-93be21a5a821 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:11.346086 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405fe88-3802-44cf-aacd-93be21a5a821 admin admin] PATH_INFO: `/v3/projects/a1740e871bae43d98be4a2509bab00eb/users/1dd1db17b81548fa98b1a3b44a3e0d53/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:11.366780 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a405fe88-3802-44cf-aacd-93be21a5a821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1740e871bae43d98be4a2509bab00eb, user_id=1dd1db17b81548fa98b1a3b44a3e0d53, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:11.370370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a405fe88-3802-44cf-aacd-93be21a5a821 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:11.388312 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1285/2540] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:55:11 2022] PUT /identity/v3/projects/a1740e871bae43d98be4a2509bab00eb/users/1dd1db17b81548fa98b1a3b44a3e0d53/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:55:11.398575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe34bb-4bcb-44c5-9d25-ce2552c7a861 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:11.398845 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe34bb-4bcb-44c5-9d25-ce2552c7a861 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:11.399118 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe34bb-4bcb-44c5-9d25-ce2552c7a861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:11.434317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-3dbe34bb-4bcb-44c5-9d25-ce2552c7a861 None None] MFA Rules not processed for user `1dd1db17b81548fa98b1a3b44a3e0d53`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:11.435623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3dbe34bb-4bcb-44c5-9d25-ce2552c7a861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:11.487808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:11.604387 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1256/2541] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:55:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:55:11.619892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405fe88-3802-44cf-aacd-93be21a5a821 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:11.622165 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-a405fe88-3802-44cf-aacd-93be21a5a821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:11.719616 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87f7d3-5355-4687-933e-7b8b47324c31 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:11.720285 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87f7d3-5355-4687-933e-7b8b47324c31 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:11.720512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87f7d3-5355-4687-933e-7b8b47324c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:11.753160 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405fe88-3802-44cf-aacd-93be21a5a821 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:11.756474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-4e87f7d3-5355-4687-933e-7b8b47324c31 None None] MFA Rules not processed for user `6655366e7eb84a26aa3f7fbe88af62b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:11.759582 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4e87f7d3-5355-4687-933e-7b8b47324c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:11.761153 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344fc29-4218-4ce2-8a1d-1e27919e6892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:11.763568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344fc29-4218-4ce2-8a1d-1e27919e6892 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:11.763863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344fc29-4218-4ce2-8a1d-1e27919e6892 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:11.764090 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344fc29-4218-4ce2-8a1d-1e27919e6892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:11.793766 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344fc29-4218-4ce2-8a1d-1e27919e6892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:11.794951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344fc29-4218-4ce2-8a1d-1e27919e6892 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:11.810266 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1286/2542] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:55:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:55:11.850486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344fc29-4218-4ce2-8a1d-1e27919e6892 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:11.957424 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1257/2543] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jun 27 15:55:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:11.976562 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344fc29-4218-4ce2-8a1d-1e27919e6892 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:11.979427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87f7d3-5355-4687-933e-7b8b47324c31 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:11.981450 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4e87f7d3-5355-4687-933e-7b8b47324c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:11.983898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a289806-40cf-41bc-94f1-17243bd02bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:11.984956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a289806-40cf-41bc-94f1-17243bd02bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:11.985353 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a289806-40cf-41bc-94f1-17243bd02bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:11.985784 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a289806-40cf-41bc-94f1-17243bd02bf1 admin admin] PATH_INFO: `/v3/users/1dd1db17b81548fa98b1a3b44a3e0d53` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:11.987308 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a289806-40cf-41bc-94f1-17243bd02bf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dd1db17b81548fa98b1a3b44a3e0d53)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:11.989878 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a289806-40cf-41bc-94f1-17243bd02bf1 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:12.031879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:12.089942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5a289806-40cf-41bc-94f1-17243bd02bf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dd1db17b81548fa98b1a3b44a3e0d53'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:12.125416 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5a289806-40cf-41bc-94f1-17243bd02bf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dd1db17b81548fa98b1a3b44a3e0d53'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:12.126165 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87f7d3-5355-4687-933e-7b8b47324c31 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:12.126655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5a289806-40cf-41bc-94f1-17243bd02bf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dd1db17b81548fa98b1a3b44a3e0d53'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:12.142214 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb1079-f2e0-428b-822a-46aee523359c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e83ea69ded040448fe75caa221ad377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:12.143562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb1079-f2e0-428b-822a-46aee523359c service swift] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:12.144124 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb1079-f2e0-428b-822a-46aee523359c service swift] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:12.144682 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb1079-f2e0-428b-822a-46aee523359c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:12.147427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5a289806-40cf-41bc-94f1-17243bd02bf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dd1db17b81548fa98b1a3b44a3e0d53'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:12.150395 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d4cb1079-f2e0-428b-822a-46aee523359c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:12.172204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5a289806-40cf-41bc-94f1-17243bd02bf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dd1db17b81548fa98b1a3b44a3e0d53'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:12.182237 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb1079-f2e0-428b-822a-46aee523359c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:12.186621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb1079-f2e0-428b-822a-46aee523359c service swift] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:12.193420 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1287/2544] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:55:11 2022] DELETE /identity/v3/users/1dd1db17b81548fa98b1a3b44a3e0d53 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:55:12.204764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10888b3-54ae-48d7-9d62-51f7f06902c2 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:12.205410 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10888b3-54ae-48d7-9d62-51f7f06902c2 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:12.205980 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10888b3-54ae-48d7-9d62-51f7f06902c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:12.211305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1258/2545] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jun 27 15:55:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 15:55:12.277289 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-e10888b3-54ae-48d7-9d62-51f7f06902c2 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:12.278583 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e10888b3-54ae-48d7-9d62-51f7f06902c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:12.503263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1288/2546] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:12.521856 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb1079-f2e0-428b-822a-46aee523359c service swift] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:12.525519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d4cb1079-f2e0-428b-822a-46aee523359c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:12.681044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb1079-f2e0-428b-822a-46aee523359c service swift] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:12.723841 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:12.846507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1caedc3-527c-48ad-ac1d-d78e284a785d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:12.847628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caedc3-527c-48ad-ac1d-d78e284a785d service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:12.848269 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caedc3-527c-48ad-ac1d-d78e284a785d service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:12.848852 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1caedc3-527c-48ad-ac1d-d78e284a785d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:12.895644 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1caedc3-527c-48ad-ac1d-d78e284a785d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:12.898461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1caedc3-527c-48ad-ac1d-d78e284a785d service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:12.925357 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1259/2547] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:55:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:55:13.671908 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10888b3-54ae-48d7-9d62-51f7f06902c2 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:13.675233 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e10888b3-54ae-48d7-9d62-51f7f06902c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:13.746802 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10888b3-54ae-48d7-9d62-51f7f06902c2 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:13.772869 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769f218-0899-4ed7-8468-aae2d61b7c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:13.775026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769f218-0899-4ed7-8468-aae2d61b7c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:13.775557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769f218-0899-4ed7-8468-aae2d61b7c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:13.776075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769f218-0899-4ed7-8468-aae2d61b7c52 admin admin] PATH_INFO: `/v3/projects/a1740e871bae43d98be4a2509bab00eb` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:13.778499 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769f218-0899-4ed7-8468-aae2d61b7c52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1740e871bae43d98be4a2509bab00eb)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:13.781971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769f218-0899-4ed7-8468-aae2d61b7c52 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:13.947912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9769f218-0899-4ed7-8468-aae2d61b7c52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1740e871bae43d98be4a2509bab00eb', 'request_id': 'req-9769f218-0899-4ed7-8468-aae2d61b7c52'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:13.962201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9769f218-0899-4ed7-8468-aae2d61b7c52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1740e871bae43d98be4a2509bab00eb', 'request_id': 'req-9769f218-0899-4ed7-8468-aae2d61b7c52'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:13.984150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9769f218-0899-4ed7-8468-aae2d61b7c52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1740e871bae43d98be4a2509bab00eb', 'request_id': 'req-9769f218-0899-4ed7-8468-aae2d61b7c52'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:14.012400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9769f218-0899-4ed7-8468-aae2d61b7c52 admin admin] The token cache is being invalidate because project a1740e871bae43d98be4a2509bab00eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:55:14.014514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9769f218-0899-4ed7-8468-aae2d61b7c52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:14.017748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9769f218-0899-4ed7-8468-aae2d61b7c52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:14.043270 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1289/2548] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:55:13 2022] DELETE /identity/v3/projects/a1740e871bae43d98be4a2509bab00eb => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:55:14.052040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a54be6-f23c-426c-b669-cbdbceb8a7ac None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:14.052992 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a54be6-f23c-426c-b669-cbdbceb8a7ac None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:14.053848 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a54be6-f23c-426c-b669-cbdbceb8a7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:14.098330 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-f6a54be6-f23c-426c-b669-cbdbceb8a7ac None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:14.099957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f6a54be6-f23c-426c-b669-cbdbceb8a7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:14.319667 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1260/2549] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:55:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:55:14.331347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e6d3a-b114-40ea-b1e5-3e5b2359d628 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:14.331938 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e6d3a-b114-40ea-b1e5-3e5b2359d628 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:14.332230 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e6d3a-b114-40ea-b1e5-3e5b2359d628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:14.397279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-d57e6d3a-b114-40ea-b1e5-3e5b2359d628 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:14.400191 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-d57e6d3a-b114-40ea-b1e5-3e5b2359d628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:14.428293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1290/2550] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:55:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:14.430681 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a54be6-f23c-426c-b669-cbdbceb8a7ac None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:14.458535 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a54be6-f23c-426c-b669-cbdbceb8a7ac None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:14.480425 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be1893-5424-4fef-aece-0071722db167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:14.490377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be1893-5424-4fef-aece-0071722db167 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:14.491213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be1893-5424-4fef-aece-0071722db167 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:14.491213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be1893-5424-4fef-aece-0071722db167 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:14.493850 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be1893-5424-4fef-aece-0071722db167 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:55:14.494412 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be1893-5424-4fef-aece-0071722db167 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:14.502035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be1893-5424-4fef-aece-0071722db167 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:14.582625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1261/2551] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:55:14.591521 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e6d3a-b114-40ea-b1e5-3e5b2359d628 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:14.605969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e6d3a-b114-40ea-b1e5-3e5b2359d628 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:14.624701 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec420f1-0b0d-40ed-9ca5-d64f3a9bd8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:14.626874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec420f1-0b0d-40ed-9ca5-d64f3a9bd8af admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:14.627728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec420f1-0b0d-40ed-9ca5-d64f3a9bd8af admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:14.628680 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec420f1-0b0d-40ed-9ca5-d64f3a9bd8af admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:14.644150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec420f1-0b0d-40ed-9ca5-d64f3a9bd8af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:14.654708 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec420f1-0b0d-40ed-9ca5-d64f3a9bd8af admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:14.742487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:14.777382 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1291/2552] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:55:14 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:55:14.789114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be1893-5424-4fef-aece-0071722db167 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:14.809543 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be1893-5424-4fef-aece-0071722db167 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:15.188763 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e6f8a-7661-4587-baa6-ed92c0e93c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:15.189950 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e6f8a-7661-4587-baa6-ed92c0e93c3c admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:15.190433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e6f8a-7661-4587-baa6-ed92c0e93c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:15.190917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e6f8a-7661-4587-baa6-ed92c0e93c3c admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:15.192181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e6f8a-7661-4587-baa6-ed92c0e93c3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:15.213071 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e6f8a-7661-4587-baa6-ed92c0e93c3c admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:15.308741 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1262/2553] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:55:14 2022] POST /identity/v3/users => generated 293 bytes in 520 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:55:15.314020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec420f1-0b0d-40ed-9ca5-d64f3a9bd8af admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:15.349078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec420f1-0b0d-40ed-9ca5-d64f3a9bd8af admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:15.381655 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afe815-670a-4a92-bf33-8473feb55039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:15.382610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afe815-670a-4a92-bf33-8473feb55039 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:15.383093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afe815-670a-4a92-bf33-8473feb55039 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:15.385071 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afe815-670a-4a92-bf33-8473feb55039 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:15.386465 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afe815-670a-4a92-bf33-8473feb55039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:15.391997 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afe815-670a-4a92-bf33-8473feb55039 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:15.425280 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1292/2554] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:55:15.429979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e6f8a-7661-4587-baa6-ed92c0e93c3c admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:15.468222 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e6f8a-7661-4587-baa6-ed92c0e93c3c admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:15.519808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac9320-317d-4ea3-8a0c-cfa27abd5966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:15.521397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac9320-317d-4ea3-8a0c-cfa27abd5966 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:15.522239 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac9320-317d-4ea3-8a0c-cfa27abd5966 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:15.530214 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac9320-317d-4ea3-8a0c-cfa27abd5966 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:15.531776 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ac9320-317d-4ea3-8a0c-cfa27abd5966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:15.534927 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ac9320-317d-4ea3-8a0c-cfa27abd5966 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:15.585754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1263/2555] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:55:15.588747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afe815-670a-4a92-bf33-8473feb55039 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:15.620282 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afe815-670a-4a92-bf33-8473feb55039 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:15.638785 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b29368-cd60-448d-9617-422313285a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:15.640056 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b29368-cd60-448d-9617-422313285a99 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:15.640356 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b29368-cd60-448d-9617-422313285a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:15.640623 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b29368-cd60-448d-9617-422313285a99 admin admin] PATH_INFO: `/v3/projects/03ad64e6cb5744e3affe1526eb73f2aa/users/df1e40c1b8fe4d80bba8597c47068c2c/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:15.672856 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b29368-cd60-448d-9617-422313285a99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03ad64e6cb5744e3affe1526eb73f2aa, user_id=df1e40c1b8fe4d80bba8597c47068c2c, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:15.676175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b29368-cd60-448d-9617-422313285a99 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:15.710799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1293/2556] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:55:15 2022] PUT /identity/v3/projects/03ad64e6cb5744e3affe1526eb73f2aa/users/df1e40c1b8fe4d80bba8597c47068c2c/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:55:15.730883 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa921d-a457-48d4-916c-fb1876d1f2ed None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:15.731306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa921d-a457-48d4-916c-fb1876d1f2ed None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:15.731469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa921d-a457-48d4-916c-fb1876d1f2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:15.823591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-beaa921d-a457-48d4-916c-fb1876d1f2ed None None] MFA Rules not processed for user `df1e40c1b8fe4d80bba8597c47068c2c`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:15.831138 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-beaa921d-a457-48d4-916c-fb1876d1f2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:15.878195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:16.046439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1264/2557] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:55:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:55:16.060177 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b29368-cd60-448d-9617-422313285a99 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:16.062351 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-d0b29368-cd60-448d-9617-422313285a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:16.202397 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b29368-cd60-448d-9617-422313285a99 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:16.208304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669143a-7a21-44c3-98b1-f8de06f169a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:16.209361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669143a-7a21-44c3-98b1-f8de06f169a6 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:16.209774 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669143a-7a21-44c3-98b1-f8de06f169a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:16.210057 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669143a-7a21-44c3-98b1-f8de06f169a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:16.224212 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669143a-7a21-44c3-98b1-f8de06f169a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:16.225754 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669143a-7a21-44c3-98b1-f8de06f169a6 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:16.241542 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1294/2558] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:55:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:55:23.182946 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa921d-a457-48d4-916c-fb1876d1f2ed None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:23.185843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-beaa921d-a457-48d4-916c-fb1876d1f2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:23.338654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa921d-a457-48d4-916c-fb1876d1f2ed None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:23.349153 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b1468-c1fc-4159-9990-8cc3944fcad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:23.350846 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b1468-c1fc-4159-9990-8cc3944fcad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:23.351331 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b1468-c1fc-4159-9990-8cc3944fcad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:23.352422 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b1468-c1fc-4159-9990-8cc3944fcad5 admin admin] PATH_INFO: `/v3/users/1320328d4bd04dc28c0c828a0f30e641` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:23.355085 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b1468-c1fc-4159-9990-8cc3944fcad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1320328d4bd04dc28c0c828a0f30e641)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:23.360308 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b1468-c1fc-4159-9990-8cc3944fcad5 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:23.463420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4e3b1468-c1fc-4159-9990-8cc3944fcad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1320328d4bd04dc28c0c828a0f30e641'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:23.507354 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4e3b1468-c1fc-4159-9990-8cc3944fcad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1320328d4bd04dc28c0c828a0f30e641'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:23.528366 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4e3b1468-c1fc-4159-9990-8cc3944fcad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1320328d4bd04dc28c0c828a0f30e641'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:23.531503 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4e3b1468-c1fc-4159-9990-8cc3944fcad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1320328d4bd04dc28c0c828a0f30e641'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:23.535641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4e3b1468-c1fc-4159-9990-8cc3944fcad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1320328d4bd04dc28c0c828a0f30e641'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:23.582333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1265/2559] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:55:23 2022] DELETE /identity/v3/users/1320328d4bd04dc28c0c828a0f30e641 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:55:23.979029 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669143a-7a21-44c3-98b1-f8de06f169a6 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:23.998460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669143a-7a21-44c3-98b1-f8de06f169a6 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:24.141494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ca964-15e2-498e-924d-8642db792574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:24.142436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ca964-15e2-498e-924d-8642db792574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:24.142725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ca964-15e2-498e-924d-8642db792574 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:24.143009 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ca964-15e2-498e-924d-8642db792574 admin admin] PATH_INFO: `/v3/projects/6a070d552fee4594bbcf0ae7adc16f94` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:24.144417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ca964-15e2-498e-924d-8642db792574 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a070d552fee4594bbcf0ae7adc16f94)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:24.146772 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ca964-15e2-498e-924d-8642db792574 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:24.227585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b19ca964-15e2-498e-924d-8642db792574 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a070d552fee4594bbcf0ae7adc16f94', 'request_id': 'req-b19ca964-15e2-498e-924d-8642db792574'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:24.235827 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b19ca964-15e2-498e-924d-8642db792574 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a070d552fee4594bbcf0ae7adc16f94', 'request_id': 'req-b19ca964-15e2-498e-924d-8642db792574'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:24.266168 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b19ca964-15e2-498e-924d-8642db792574 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a070d552fee4594bbcf0ae7adc16f94', 'request_id': 'req-b19ca964-15e2-498e-924d-8642db792574'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:24.294068 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:24.322939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b19ca964-15e2-498e-924d-8642db792574 admin admin] The token cache is being invalidate because project 6a070d552fee4594bbcf0ae7adc16f94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:55:24.323447 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b19ca964-15e2-498e-924d-8642db792574 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:24.326299 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-b19ca964-15e2-498e-924d-8642db792574 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:24.340530 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1295/2560] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:55:23 2022] DELETE /identity/v3/projects/6a070d552fee4594bbcf0ae7adc16f94 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:55:24.341828 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b1468-c1fc-4159-9990-8cc3944fcad5 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:24.343404 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4e3b1468-c1fc-4159-9990-8cc3944fcad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:24.372334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b1468-c1fc-4159-9990-8cc3944fcad5 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:24.482673 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:24.628705 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b41f7-5c18-4603-855d-6795fa050a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:24.630242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b41f7-5c18-4603-855d-6795fa050a20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:24.630760 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b41f7-5c18-4603-855d-6795fa050a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:24.632325 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b41f7-5c18-4603-855d-6795fa050a20 admin admin] PATH_INFO: `/v3/users/7f551fe9c2d04de184de188eda5a830a` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:24.634148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b41f7-5c18-4603-855d-6795fa050a20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f551fe9c2d04de184de188eda5a830a)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:24.638346 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b41f7-5c18-4603-855d-6795fa050a20 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:24.767164 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2b1b41f7-5c18-4603-855d-6795fa050a20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f551fe9c2d04de184de188eda5a830a'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:24.788643 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2b1b41f7-5c18-4603-855d-6795fa050a20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f551fe9c2d04de184de188eda5a830a'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:24.812446 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2b1b41f7-5c18-4603-855d-6795fa050a20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f551fe9c2d04de184de188eda5a830a'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:24.813733 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2b1b41f7-5c18-4603-855d-6795fa050a20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f551fe9c2d04de184de188eda5a830a'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:24.815089 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2b1b41f7-5c18-4603-855d-6795fa050a20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f551fe9c2d04de184de188eda5a830a'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:24.851399 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1266/2561] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:55:24 2022] DELETE /identity/v3/users/7f551fe9c2d04de184de188eda5a830a => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:55:25.286253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ca964-15e2-498e-924d-8642db792574 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:25.290703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-b19ca964-15e2-498e-924d-8642db792574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:25.339417 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ca964-15e2-498e-924d-8642db792574 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:25.594935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b5fc8-f6cd-482e-aa1e-a56b64035955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:25.597520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b5fc8-f6cd-482e-aa1e-a56b64035955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:25.598529 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b5fc8-f6cd-482e-aa1e-a56b64035955 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:25.599167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b5fc8-f6cd-482e-aa1e-a56b64035955 admin admin] PATH_INFO: `/v3/projects/4a048153141b485a92cfb09dd6902a91` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:25.606190 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b5fc8-f6cd-482e-aa1e-a56b64035955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a048153141b485a92cfb09dd6902a91)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:25.610433 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b5fc8-f6cd-482e-aa1e-a56b64035955 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:25.708990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b41f7-5c18-4603-855d-6795fa050a20 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:25.716547 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-2b1b41f7-5c18-4603-855d-6795fa050a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:25.765262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b41f7-5c18-4603-855d-6795fa050a20 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:25.774244 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-7a1b5fc8-f6cd-482e-aa1e-a56b64035955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a048153141b485a92cfb09dd6902a91', 'request_id': 'req-7a1b5fc8-f6cd-482e-aa1e-a56b64035955'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:25.785069 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-7a1b5fc8-f6cd-482e-aa1e-a56b64035955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a048153141b485a92cfb09dd6902a91', 'request_id': 'req-7a1b5fc8-f6cd-482e-aa1e-a56b64035955'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:25.804379 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-7a1b5fc8-f6cd-482e-aa1e-a56b64035955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a048153141b485a92cfb09dd6902a91', 'request_id': 'req-7a1b5fc8-f6cd-482e-aa1e-a56b64035955'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:25.851550 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-7a1b5fc8-f6cd-482e-aa1e-a56b64035955 admin admin] The token cache is being invalidate because project 4a048153141b485a92cfb09dd6902a91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:55:25.852611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-7a1b5fc8-f6cd-482e-aa1e-a56b64035955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:25.857039 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-7a1b5fc8-f6cd-482e-aa1e-a56b64035955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:25.860411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1296/2562] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:55:25 2022] DELETE /identity/v3/projects/4a048153141b485a92cfb09dd6902a91 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:55:25.999977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbc390-1924-47b5-b5ee-5c909a8ab3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:26.006952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbc390-1924-47b5-b5ee-5c909a8ab3e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:26.007481 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbc390-1924-47b5-b5ee-5c909a8ab3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:26.008449 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbc390-1924-47b5-b5ee-5c909a8ab3e1 admin admin] PATH_INFO: `/v3/users/df1e40c1b8fe4d80bba8597c47068c2c` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:26.012881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbc390-1924-47b5-b5ee-5c909a8ab3e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df1e40c1b8fe4d80bba8597c47068c2c)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:26.019265 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbc390-1924-47b5-b5ee-5c909a8ab3e1 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:26.176238 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-27cbc390-1924-47b5-b5ee-5c909a8ab3e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df1e40c1b8fe4d80bba8597c47068c2c'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:26.192556 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:26.214265 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-27cbc390-1924-47b5-b5ee-5c909a8ab3e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df1e40c1b8fe4d80bba8597c47068c2c'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:26.248591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-27cbc390-1924-47b5-b5ee-5c909a8ab3e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df1e40c1b8fe4d80bba8597c47068c2c'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:26.249837 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-27cbc390-1924-47b5-b5ee-5c909a8ab3e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df1e40c1b8fe4d80bba8597c47068c2c'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:26.250998 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-27cbc390-1924-47b5-b5ee-5c909a8ab3e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df1e40c1b8fe4d80bba8597c47068c2c'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:26.295277 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1267/2563] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:55:25 2022] DELETE /identity/v3/users/df1e40c1b8fe4d80bba8597c47068c2c => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:55:26.306222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b0971-9839-449a-8f39-b025f8ac99f0 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:26.307063 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b0971-9839-449a-8f39-b025f8ac99f0 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:26.307494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b0971-9839-449a-8f39-b025f8ac99f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:26.343500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-e82b0971-9839-449a-8f39-b025f8ac99f0 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:26.345668 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e82b0971-9839-449a-8f39-b025f8ac99f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:26.489904 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1297/2564] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:26.507788 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbc390-1924-47b5-b5ee-5c909a8ab3e1 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:26.511074 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-27cbc390-1924-47b5-b5ee-5c909a8ab3e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:26.699601 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbc390-1924-47b5-b5ee-5c909a8ab3e1 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:26.713496 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ab7da-4c83-4762-a695-5382e280ee87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:26.714753 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ab7da-4c83-4762-a695-5382e280ee87 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:26.715335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ab7da-4c83-4762-a695-5382e280ee87 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:26.715890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ab7da-4c83-4762-a695-5382e280ee87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:26.745703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ab7da-4c83-4762-a695-5382e280ee87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:26.748621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ab7da-4c83-4762-a695-5382e280ee87 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:26.768478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1268/2565] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:55:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:55:27.426838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b5d5c-970c-47d6-af4b-46c49a71692a None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:27.427747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b5d5c-970c-47d6-af4b-46c49a71692a None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:27.428217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b5d5c-970c-47d6-af4b-46c49a71692a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:27.485832 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-4b5b5d5c-970c-47d6-af4b-46c49a71692a None None] MFA Rules not processed for user `ce59e0072f1c4b91a2825e5fecde7e23`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:27.493826 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-4b5b5d5c-970c-47d6-af4b-46c49a71692a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:27.509297 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ab7da-4c83-4762-a695-5382e280ee87 service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:27.512874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-656ab7da-4c83-4762-a695-5382e280ee87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:27.551071 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ab7da-4c83-4762-a695-5382e280ee87 service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:27.559989 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730cb83-136d-472a-b9d7-e13a803a65e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:27.561177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730cb83-136d-472a-b9d7-e13a803a65e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:27.561938 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730cb83-136d-472a-b9d7-e13a803a65e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:27.562093 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730cb83-136d-472a-b9d7-e13a803a65e1 admin admin] PATH_INFO: `/v3/projects/03ad64e6cb5744e3affe1526eb73f2aa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:27.565000 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730cb83-136d-472a-b9d7-e13a803a65e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03ad64e6cb5744e3affe1526eb73f2aa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:27.567980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730cb83-136d-472a-b9d7-e13a803a65e1 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:27.570091 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:27.657901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d730cb83-136d-472a-b9d7-e13a803a65e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03ad64e6cb5744e3affe1526eb73f2aa', 'request_id': 'req-d730cb83-136d-472a-b9d7-e13a803a65e1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:27.661284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1298/2566] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:55:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:27.677955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d730cb83-136d-472a-b9d7-e13a803a65e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03ad64e6cb5744e3affe1526eb73f2aa', 'request_id': 'req-d730cb83-136d-472a-b9d7-e13a803a65e1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:27.689661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d730cb83-136d-472a-b9d7-e13a803a65e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03ad64e6cb5744e3affe1526eb73f2aa', 'request_id': 'req-d730cb83-136d-472a-b9d7-e13a803a65e1'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:27.712072 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d730cb83-136d-472a-b9d7-e13a803a65e1 admin admin] The token cache is being invalidate because project 03ad64e6cb5744e3affe1526eb73f2aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:55:27.712418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d730cb83-136d-472a-b9d7-e13a803a65e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:27.713060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d730cb83-136d-472a-b9d7-e13a803a65e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:55:27.715303 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1269/2567] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:55:27 2022] DELETE /identity/v3/projects/03ad64e6cb5744e3affe1526eb73f2aa => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:55:27.725910 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37239da7-58ee-4218-a10d-ba4d8bba2113 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:27.726353 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37239da7-58ee-4218-a10d-ba4d8bba2113 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:27.727113 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37239da7-58ee-4218-a10d-ba4d8bba2113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:27.757556 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-37239da7-58ee-4218-a10d-ba4d8bba2113 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:27.759908 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-37239da7-58ee-4218-a10d-ba4d8bba2113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:27.906933 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1299/2568] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:27.920978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed129c94-70f6-4fda-b5b7-2cb272da9f1a None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:27.921487 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed129c94-70f6-4fda-b5b7-2cb272da9f1a None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:27.922004 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed129c94-70f6-4fda-b5b7-2cb272da9f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:27.955529 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-ed129c94-70f6-4fda-b5b7-2cb272da9f1a None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:27.956906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-ed129c94-70f6-4fda-b5b7-2cb272da9f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:27.966783 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1270/2569] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:27.972061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37239da7-58ee-4218-a10d-ba4d8bba2113 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:27.983375 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37239da7-58ee-4218-a10d-ba4d8bba2113 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:27.988606 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf846e2-6c84-42db-b9fd-4a60aea78968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:27.989743 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf846e2-6c84-42db-b9fd-4a60aea78968 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:27.990032 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf846e2-6c84-42db-b9fd-4a60aea78968 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:27.990464 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf846e2-6c84-42db-b9fd-4a60aea78968 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:27.991500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf846e2-6c84-42db-b9fd-4a60aea78968 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:55:27.991883 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf846e2-6c84-42db-b9fd-4a60aea78968 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:27.993842 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf846e2-6c84-42db-b9fd-4a60aea78968 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:28.009075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1300/2570] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:55:28.012775 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed129c94-70f6-4fda-b5b7-2cb272da9f1a None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:28.023268 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed129c94-70f6-4fda-b5b7-2cb272da9f1a None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:28.028954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1033533-d9d5-4844-94b7-3fe55ae6fa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:28.029669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1033533-d9d5-4844-94b7-3fe55ae6fa4e admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:28.029977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1033533-d9d5-4844-94b7-3fe55ae6fa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:28.030259 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1033533-d9d5-4844-94b7-3fe55ae6fa4e admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:28.030981 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1033533-d9d5-4844-94b7-3fe55ae6fa4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:28.033171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1033533-d9d5-4844-94b7-3fe55ae6fa4e admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:28.076301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1271/2571] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:55:28 2022] POST /identity/v3/projects => generated 361 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:55:28.078254 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf846e2-6c84-42db-b9fd-4a60aea78968 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:28.092514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf846e2-6c84-42db-b9fd-4a60aea78968 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:28.192927 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ea898-23ce-4cec-8dcc-44edfec1c68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:28.193623 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ea898-23ce-4cec-8dcc-44edfec1c68c admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:28.193939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ea898-23ce-4cec-8dcc-44edfec1c68c admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:28.194222 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ea898-23ce-4cec-8dcc-44edfec1c68c admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:28.195011 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ea898-23ce-4cec-8dcc-44edfec1c68c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:28.197836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ea898-23ce-4cec-8dcc-44edfec1c68c admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:28.224525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1301/2572] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:55:28 2022] POST /identity/v3/users => generated 281 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:55:28.229906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1033533-d9d5-4844-94b7-3fe55ae6fa4e admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:28.241077 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1033533-d9d5-4844-94b7-3fe55ae6fa4e admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:28.246296 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8df485-8ec2-4fc3-b5c0-23669ce69939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:28.247073 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8df485-8ec2-4fc3-b5c0-23669ce69939 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:28.247444 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8df485-8ec2-4fc3-b5c0-23669ce69939 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:28.248336 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8df485-8ec2-4fc3-b5c0-23669ce69939 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:28.248638 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8df485-8ec2-4fc3-b5c0-23669ce69939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:28.250376 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8df485-8ec2-4fc3-b5c0-23669ce69939 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:28.265024 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1272/2573] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:55:28.269022 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ea898-23ce-4cec-8dcc-44edfec1c68c admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:28.281457 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ea898-23ce-4cec-8dcc-44edfec1c68c admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:28.288408 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241bffb5-b929-4f38-8bc1-e4aa0c526b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:28.290220 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bffb5-b929-4f38-8bc1-e4aa0c526b5d admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:28.290220 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bffb5-b929-4f38-8bc1-e4aa0c526b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:28.290565 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bffb5-b929-4f38-8bc1-e4aa0c526b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:28.291449 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241bffb5-b929-4f38-8bc1-e4aa0c526b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:28.293081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241bffb5-b929-4f38-8bc1-e4aa0c526b5d admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:28.307476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1302/2574] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:55:28.311590 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8df485-8ec2-4fc3-b5c0-23669ce69939 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:28.323177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8df485-8ec2-4fc3-b5c0-23669ce69939 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:28.328103 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86113ef4-e0fc-42a3-9d0b-50fa7ba86baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:28.329017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86113ef4-e0fc-42a3-9d0b-50fa7ba86baf admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:28.329302 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86113ef4-e0fc-42a3-9d0b-50fa7ba86baf admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:28.329578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86113ef4-e0fc-42a3-9d0b-50fa7ba86baf admin admin] PATH_INFO: `/v3/projects/10e96141c5714b07b740e82a507317b5/users/3deeaea432b14f0a8ec8bdcbb3675398/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:28.347176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86113ef4-e0fc-42a3-9d0b-50fa7ba86baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e96141c5714b07b740e82a507317b5, user_id=3deeaea432b14f0a8ec8bdcbb3675398, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:28.349916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86113ef4-e0fc-42a3-9d0b-50fa7ba86baf admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:28.354007 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:28.381445 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1273/2575] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:55:28 2022] PUT /identity/v3/projects/10e96141c5714b07b740e82a507317b5/users/3deeaea432b14f0a8ec8bdcbb3675398/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:55:28.384213 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716d25a-e420-477e-9937-9170017d2e4c None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:28.384935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716d25a-e420-477e-9937-9170017d2e4c None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:28.385418 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716d25a-e420-477e-9937-9170017d2e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:28.432834 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-4716d25a-e420-477e-9937-9170017d2e4c None None] MFA Rules not processed for user `3deeaea432b14f0a8ec8bdcbb3675398`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:28.434578 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-4716d25a-e420-477e-9937-9170017d2e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:28.472086 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:28.527772 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1303/2576] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:55:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:28.541737 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86113ef4-e0fc-42a3-9d0b-50fa7ba86baf admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:28.543949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-86113ef4-e0fc-42a3-9d0b-50fa7ba86baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:28.642590 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86113ef4-e0fc-42a3-9d0b-50fa7ba86baf admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:28.647500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe24e16-27e4-4bc1-83eb-1ffca915d422 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:28.647904 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe24e16-27e4-4bc1-83eb-1ffca915d422 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:28.648499 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe24e16-27e4-4bc1-83eb-1ffca915d422 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:28.648991 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe24e16-27e4-4bc1-83eb-1ffca915d422 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:28.660621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe24e16-27e4-4bc1-83eb-1ffca915d422 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:28.662264 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe24e16-27e4-4bc1-83eb-1ffca915d422 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:28.674342 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1274/2577] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:55:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:55:33.780321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e96283-82bc-4f2a-b0a2-bbc0abb23cff None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:33.780998 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e96283-82bc-4f2a-b0a2-bbc0abb23cff None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:33.781258 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e96283-82bc-4f2a-b0a2-bbc0abb23cff None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:33.784041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1304/2578] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jun 27 15:55:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:55:33.792271 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9884979-c543-4829-a7f8-8be6ef309f7a None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:33.793539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9884979-c543-4829-a7f8-8be6ef309f7a None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:33.794250 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9884979-c543-4829-a7f8-8be6ef309f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:33.842822 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-d9884979-c543-4829-a7f8-8be6ef309f7a None None] MFA Rules not processed for user `1c8f6262cf24419daf803ae71aed24f5`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:33.844420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d9884979-c543-4829-a7f8-8be6ef309f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:33.947930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1275/2579] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jun 27 15:55:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:33.957260 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e96283-82bc-4f2a-b0a2-bbc0abb23cff None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:33.959534 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-a1e96283-82bc-4f2a-b0a2-bbc0abb23cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:34.055549 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e96283-82bc-4f2a-b0a2-bbc0abb23cff None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:34.060436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f84a0-0f8b-4e40-ac6d-ec7d074a8902 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493fcde4a45944efbc0b52c44e61b411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:34.062042 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f84a0-0f8b-4e40-ac6d-ec7d074a8902 service glance] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:34.062735 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f84a0-0f8b-4e40-ac6d-ec7d074a8902 service glance] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:34.063167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f84a0-0f8b-4e40-ac6d-ec7d074a8902 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:34.082889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f84a0-0f8b-4e40-ac6d-ec7d074a8902 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:34.085593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f84a0-0f8b-4e40-ac6d-ec7d074a8902 service glance] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:34.102030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1305/2580] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:55:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:55:34.188144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc55923-ccf4-495f-914e-79001a96b7a2 None None] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:34.188526 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc55923-ccf4-495f-914e-79001a96b7a2 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:34.188882 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc55923-ccf4-495f-914e-79001a96b7a2 None None] PATH_INFO: `/` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:34.190962 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1276/2581] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jun 27 15:55:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:55:34.197051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a5f02-3f13-4d87-8f8c-5661d6bc3d84 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:34.197342 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a5f02-3f13-4d87-8f8c-5661d6bc3d84 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:34.197624 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a5f02-3f13-4d87-8f8c-5661d6bc3d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:34.217839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-531a5f02-3f13-4d87-8f8c-5661d6bc3d84 None None] MFA Rules not processed for user `42dee639e2834d99b7468f911c37cf8f`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:34.219097 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-531a5f02-3f13-4d87-8f8c-5661d6bc3d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:34.289953 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1306/2582] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jun 27 15:55:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:34.306630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc55923-ccf4-495f-914e-79001a96b7a2 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:34.308844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-9cc55923-ccf4-495f-914e-79001a96b7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:34.324748 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc55923-ccf4-495f-914e-79001a96b7a2 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:34.328538 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3750e-c7fa-4eb3-b00a-a59d3cbfe66b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:34.329213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3750e-c7fa-4eb3-b00a-a59d3cbfe66b service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:34.329499 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3750e-c7fa-4eb3-b00a-a59d3cbfe66b service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:34.329813 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3750e-c7fa-4eb3-b00a-a59d3cbfe66b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:34.339095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3750e-c7fa-4eb3-b00a-a59d3cbfe66b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:34.340568 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3750e-c7fa-4eb3-b00a-a59d3cbfe66b service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:34.351051 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1277/2583] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:55:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:55:34.448338 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ef1c9-c642-4164-9171-b71f389eb24f None None] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:34.448625 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ef1c9-c642-4164-9171-b71f389eb24f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:34.448910 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ef1c9-c642-4164-9171-b71f389eb24f None None] PATH_INFO: `/` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:34.450629 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1307/2584] 10.222.0.20 () {54 vars in 921 bytes} [Mon Jun 27 15:55:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 15:55:34.456436 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb96b4-a4ca-45e2-a8ac-b5032a8ad2ad None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:34.456714 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb96b4-a4ca-45e2-a8ac-b5032a8ad2ad None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:34.456990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb96b4-a4ca-45e2-a8ac-b5032a8ad2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:34.477571 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-fbbb96b4-a4ca-45e2-a8ac-b5032a8ad2ad None None] MFA Rules not processed for user `1c8f6262cf24419daf803ae71aed24f5`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:34.478840 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-fbbb96b4-a4ca-45e2-a8ac-b5032a8ad2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:34.489724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1278/2585] 10.222.0.20 () {58 vars in 1018 bytes} [Mon Jun 27 15:55:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:34.501674 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ef1c9-c642-4164-9171-b71f389eb24f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:34.513946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ef1c9-c642-4164-9171-b71f389eb24f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:34.517593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4a4da-e4f8-4ba6-8932-2179ae564338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:34.518286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4a4da-e4f8-4ba6-8932-2179ae564338 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:34.518575 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4a4da-e4f8-4ba6-8932-2179ae564338 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:34.518851 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4a4da-e4f8-4ba6-8932-2179ae564338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:34.532023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f4a4da-e4f8-4ba6-8932-2179ae564338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:34.535732 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f4a4da-e4f8-4ba6-8932-2179ae564338 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:34.562837 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1308/2586] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:55:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:55:36.975230 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb2c20-d94e-46e9-979b-9dbd0cbc5d48 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:36.975770 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb2c20-d94e-46e9-979b-9dbd0cbc5d48 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:36.975770 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb2c20-d94e-46e9-979b-9dbd0cbc5d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:36.996032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-9eeb2c20-d94e-46e9-979b-9dbd0cbc5d48 None None] MFA Rules not processed for user `6655366e7eb84a26aa3f7fbe88af62b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:36.997392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-9eeb2c20-d94e-46e9-979b-9dbd0cbc5d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:37.058980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1279/2587] 10.222.0.20 () {58 vars in 1009 bytes} [Mon Jun 27 15:55:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:37.072047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4a4da-e4f8-4ba6-8932-2179ae564338 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:37.073482 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-80f4a4da-e4f8-4ba6-8932-2179ae564338 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:37.154763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4a4da-e4f8-4ba6-8932-2179ae564338 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:37.160051 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce3c97-79f4-4a13-8656-e27e3f793ff4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e83ea69ded040448fe75caa221ad377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:37.160725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce3c97-79f4-4a13-8656-e27e3f793ff4 service swift] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:37.161007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce3c97-79f4-4a13-8656-e27e3f793ff4 service swift] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:37.161315 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce3c97-79f4-4a13-8656-e27e3f793ff4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:37.168862 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=100 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:37.173703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce3c97-79f4-4a13-8656-e27e3f793ff4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:37.175677 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce3c97-79f4-4a13-8656-e27e3f793ff4 service swift] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:37.189202 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1309/2588] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jun 27 15:55:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:55:47.051344 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=93 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:47.184801 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:55:53.970178 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afdc33-8887-4594-bc6a-38dda0eaa6e6 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:53.971147 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afdc33-8887-4594-bc6a-38dda0eaa6e6 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:53.971147 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afdc33-8887-4594-bc6a-38dda0eaa6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:54.004007 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-82afdc33-8887-4594-bc6a-38dda0eaa6e6 None None] MFA Rules not processed for user `6655366e7eb84a26aa3f7fbe88af62b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:55:54.006275 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-82afdc33-8887-4594-bc6a-38dda0eaa6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:54.015303 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1280/2589] 10.222.0.20 () {58 vars in 967 bytes} [Mon Jun 27 15:55:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:55:54.029347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce3c97-79f4-4a13-8656-e27e3f793ff4 service swift] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:54.051749 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce3c97-79f4-4a13-8656-e27e3f793ff4 service swift] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:54.061942 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625de944-5c79-4bcd-b376-c8ee8f2c8333 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e83ea69ded040448fe75caa221ad377', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:54.063107 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625de944-5c79-4bcd-b376-c8ee8f2c8333 service swift] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:54.063615 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625de944-5c79-4bcd-b376-c8ee8f2c8333 service swift] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:54.064128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625de944-5c79-4bcd-b376-c8ee8f2c8333 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:54.082700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625de944-5c79-4bcd-b376-c8ee8f2c8333 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:54.087024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625de944-5c79-4bcd-b376-c8ee8f2c8333 service swift] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:54.104211 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1310/2590] 10.222.0.20 () {60 vars in 1384 bytes} [Mon Jun 27 15:55:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 15:55:58.632746 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afdc33-8887-4594-bc6a-38dda0eaa6e6 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:55:58.634335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-82afdc33-8887-4594-bc6a-38dda0eaa6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:58.648580 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afdc33-8887-4594-bc6a-38dda0eaa6e6 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:55:58.652542 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef76e56-dc18-4671-9a0c-aa2ea4247d7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:55:58.653218 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef76e56-dc18-4671-9a0c-aa2ea4247d7c service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:55:58.653500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef76e56-dc18-4671-9a0c-aa2ea4247d7c service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:55:58.653829 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef76e56-dc18-4671-9a0c-aa2ea4247d7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:55:58.656046 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-aef76e56-dc18-4671-9a0c-aa2ea4247d7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:55:58.745628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef76e56-dc18-4671-9a0c-aa2ea4247d7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:55:58.747398 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef76e56-dc18-4671-9a0c-aa2ea4247d7c service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:55:58.760563 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1281/2591] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:55:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:56:04.098975 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=6 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:56:06.947823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb892204-def0-4dca-9e86-82c30351a578 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:06.948912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb892204-def0-4dca-9e86-82c30351a578 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:06.948912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb892204-def0-4dca-9e86-82c30351a578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:06.969139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-bb892204-def0-4dca-9e86-82c30351a578 None None] MFA Rules not processed for user `25024a505319480ab00f4c7e58de600c`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:56:06.970447 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-bb892204-def0-4dca-9e86-82c30351a578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:06.979204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1311/2592] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:56:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:56:08.465691 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef76e56-dc18-4671-9a0c-aa2ea4247d7c service cinder] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:08.468143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-aef76e56-dc18-4671-9a0c-aa2ea4247d7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:08.587629 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef76e56-dc18-4671-9a0c-aa2ea4247d7c service cinder] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:08.594334 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5618c0-ea4b-461a-9cfc-1b9b2d878a17 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffafb68f1bb941588a5217ad83ad907c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:08.595151 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5618c0-ea4b-461a-9cfc-1b9b2d878a17 service placement] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:08.595500 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5618c0-ea4b-461a-9cfc-1b9b2d878a17 service placement] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:08.595941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5618c0-ea4b-461a-9cfc-1b9b2d878a17 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:08.598346 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-ea5618c0-ea4b-461a-9cfc-1b9b2d878a17 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:08.613681 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5618c0-ea4b-461a-9cfc-1b9b2d878a17 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:08.615410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5618c0-ea4b-461a-9cfc-1b9b2d878a17 service placement] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:08.627983 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1282/2593] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:56:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:56:09.823239 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb892204-def0-4dca-9e86-82c30351a578 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:09.905971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb892204-def0-4dca-9e86-82c30351a578 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:09.910903 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda4212-8792-4350-b02c-8c9913694cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:09.912008 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda4212-8792-4350-b02c-8c9913694cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:09.912291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda4212-8792-4350-b02c-8c9913694cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:09.912598 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda4212-8792-4350-b02c-8c9913694cc6 admin admin] PATH_INFO: `/v3/users/3deeaea432b14f0a8ec8bdcbb3675398` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:09.913962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda4212-8792-4350-b02c-8c9913694cc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3deeaea432b14f0a8ec8bdcbb3675398)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:09.915977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda4212-8792-4350-b02c-8c9913694cc6 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:09.977865 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4bda4212-8792-4350-b02c-8c9913694cc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3deeaea432b14f0a8ec8bdcbb3675398'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:09.997567 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4bda4212-8792-4350-b02c-8c9913694cc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3deeaea432b14f0a8ec8bdcbb3675398'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:09.998466 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4bda4212-8792-4350-b02c-8c9913694cc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3deeaea432b14f0a8ec8bdcbb3675398'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:10.008883 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4bda4212-8792-4350-b02c-8c9913694cc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3deeaea432b14f0a8ec8bdcbb3675398'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:10.019740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4bda4212-8792-4350-b02c-8c9913694cc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3deeaea432b14f0a8ec8bdcbb3675398'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:10.028758 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1312/2594] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:56:09 2022] DELETE /identity/v3/users/3deeaea432b14f0a8ec8bdcbb3675398 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:56:10.033772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da3999-ba2f-4427-bd40-bc06cbadd778 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:10.034362 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da3999-ba2f-4427-bd40-bc06cbadd778 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:10.034362 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da3999-ba2f-4427-bd40-bc06cbadd778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:10.055627 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-46da3999-ba2f-4427-bd40-bc06cbadd778 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:56:10.056902 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-46da3999-ba2f-4427-bd40-bc06cbadd778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:10.168392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1283/2595] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:56:10.181293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda4212-8792-4350-b02c-8c9913694cc6 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:10.184376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-4bda4212-8792-4350-b02c-8c9913694cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:10.301493 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda4212-8792-4350-b02c-8c9913694cc6 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:10.309832 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1099c-29bf-49ac-a493-0f0b9de17a96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:10.310879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1099c-29bf-49ac-a493-0f0b9de17a96 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:10.311323 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1099c-29bf-49ac-a493-0f0b9de17a96 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:10.311768 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1099c-29bf-49ac-a493-0f0b9de17a96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:10.330379 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a1099c-29bf-49ac-a493-0f0b9de17a96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:10.333240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a1099c-29bf-49ac-a493-0f0b9de17a96 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:10.362413 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1313/2596] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:56:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:56:11.219823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da3999-ba2f-4427-bd40-bc06cbadd778 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:11.221472 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-46da3999-ba2f-4427-bd40-bc06cbadd778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:11.235792 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da3999-ba2f-4427-bd40-bc06cbadd778 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:11.240834 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11445a1b-4edb-44b8-b015-2967fc4ba197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:11.241879 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11445a1b-4edb-44b8-b015-2967fc4ba197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:11.242094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11445a1b-4edb-44b8-b015-2967fc4ba197 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:11.242560 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11445a1b-4edb-44b8-b015-2967fc4ba197 admin admin] PATH_INFO: `/v3/projects/10e96141c5714b07b740e82a507317b5` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:11.244519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11445a1b-4edb-44b8-b015-2967fc4ba197 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10e96141c5714b07b740e82a507317b5)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:11.246776 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11445a1b-4edb-44b8-b015-2967fc4ba197 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:11.278550 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:56:11.324183 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-11445a1b-4edb-44b8-b015-2967fc4ba197 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10e96141c5714b07b740e82a507317b5', 'request_id': 'req-11445a1b-4edb-44b8-b015-2967fc4ba197'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:11.335593 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-11445a1b-4edb-44b8-b015-2967fc4ba197 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10e96141c5714b07b740e82a507317b5', 'request_id': 'req-11445a1b-4edb-44b8-b015-2967fc4ba197'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:11.345390 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-11445a1b-4edb-44b8-b015-2967fc4ba197 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10e96141c5714b07b740e82a507317b5', 'request_id': 'req-11445a1b-4edb-44b8-b015-2967fc4ba197'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:11.363720 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-11445a1b-4edb-44b8-b015-2967fc4ba197 admin admin] The token cache is being invalidate because project 10e96141c5714b07b740e82a507317b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:56:11.364013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-11445a1b-4edb-44b8-b015-2967fc4ba197 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:11.364701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-11445a1b-4edb-44b8-b015-2967fc4ba197 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:11.366525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1284/2597] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:56:11 2022] DELETE /identity/v3/projects/10e96141c5714b07b740e82a507317b5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:56:11.372885 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea61f15-3811-4346-9c84-6830684057ee None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:11.373568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea61f15-3811-4346-9c84-6830684057ee None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:11.373969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea61f15-3811-4346-9c84-6830684057ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:11.404007 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-3ea61f15-3811-4346-9c84-6830684057ee None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:56:11.407267 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-3ea61f15-3811-4346-9c84-6830684057ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:11.422666 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:56:11.521979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1314/2598] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:56:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:56:11.530349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27e7d7-d72c-4eaf-a236-398d55834d97 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:11.530857 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27e7d7-d72c-4eaf-a236-398d55834d97 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:11.531364 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27e7d7-d72c-4eaf-a236-398d55834d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:11.571438 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-0d27e7d7-d72c-4eaf-a236-398d55834d97 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:56:11.572729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-0d27e7d7-d72c-4eaf-a236-398d55834d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:11.582896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1285/2599] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:56:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:56:11.587578 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea61f15-3811-4346-9c84-6830684057ee None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:11.605277 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea61f15-3811-4346-9c84-6830684057ee None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:11.621824 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a976c-3d29-49b9-bc5e-789dab73b5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:11.626316 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a976c-3d29-49b9-bc5e-789dab73b5c5 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:11.628102 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a976c-3d29-49b9-bc5e-789dab73b5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:11.628799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a976c-3d29-49b9-bc5e-789dab73b5c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:11.630898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a976c-3d29-49b9-bc5e-789dab73b5c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:56:11.632827 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a976c-3d29-49b9-bc5e-789dab73b5c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:11.637155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a976c-3d29-49b9-bc5e-789dab73b5c5 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:11.666596 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1315/2600] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:56:11.668536 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27e7d7-d72c-4eaf-a236-398d55834d97 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:11.688575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27e7d7-d72c-4eaf-a236-398d55834d97 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:11.701583 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e52acf-82a1-48c5-9fc4-bfacbcbff0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:11.703109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e52acf-82a1-48c5-9fc4-bfacbcbff0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:11.703392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e52acf-82a1-48c5-9fc4-bfacbcbff0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:11.703858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e52acf-82a1-48c5-9fc4-bfacbcbff0d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:11.704759 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e52acf-82a1-48c5-9fc4-bfacbcbff0d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:11.708135 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e52acf-82a1-48c5-9fc4-bfacbcbff0d9 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:11.746101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1286/2601] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:56:11 2022] POST /identity/v3/projects => generated 393 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:56:11.750722 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a976c-3d29-49b9-bc5e-789dab73b5c5 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:11.762636 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a976c-3d29-49b9-bc5e-789dab73b5c5 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:11.871445 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986f929-82c3-4870-b7bb-e7d9f5802def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:11.872600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986f929-82c3-4870-b7bb-e7d9f5802def admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:11.873102 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986f929-82c3-4870-b7bb-e7d9f5802def admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:11.873579 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986f929-82c3-4870-b7bb-e7d9f5802def admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:11.874934 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986f929-82c3-4870-b7bb-e7d9f5802def admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:11.878290 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986f929-82c3-4870-b7bb-e7d9f5802def admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:11.915110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1316/2602] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:56:11 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:56:11.919755 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e52acf-82a1-48c5-9fc4-bfacbcbff0d9 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:11.942366 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e52acf-82a1-48c5-9fc4-bfacbcbff0d9 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:11.951409 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3b9a2-2bc8-4676-bf81-3c7ade75c793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:11.952536 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3b9a2-2bc8-4676-bf81-3c7ade75c793 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:11.953589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3b9a2-2bc8-4676-bf81-3c7ade75c793 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:11.953589 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3b9a2-2bc8-4676-bf81-3c7ade75c793 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:11.954695 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3b9a2-2bc8-4676-bf81-3c7ade75c793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:11.961335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3b9a2-2bc8-4676-bf81-3c7ade75c793 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:11.992861 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1287/2603] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:56:11.997561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986f929-82c3-4870-b7bb-e7d9f5802def admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:12.014437 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986f929-82c3-4870-b7bb-e7d9f5802def admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:12.028209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162117c-a120-476b-81c2-f350081d9c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:12.028930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162117c-a120-476b-81c2-f350081d9c08 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:12.029219 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162117c-a120-476b-81c2-f350081d9c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:12.029498 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162117c-a120-476b-81c2-f350081d9c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:12.030323 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a162117c-a120-476b-81c2-f350081d9c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:12.032501 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a162117c-a120-476b-81c2-f350081d9c08 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:12.046450 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1317/2604] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:56:12.051594 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3b9a2-2bc8-4676-bf81-3c7ade75c793 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:12.066550 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3b9a2-2bc8-4676-bf81-3c7ade75c793 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:12.081065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62608a6d-4d15-4df1-8d80-377b19ebe667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:12.083833 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62608a6d-4d15-4df1-8d80-377b19ebe667 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:12.084190 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62608a6d-4d15-4df1-8d80-377b19ebe667 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:12.084904 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62608a6d-4d15-4df1-8d80-377b19ebe667 admin admin] PATH_INFO: `/v3/projects/b4cbbf5be5e34be398d00dc3337b2e2b/users/e50b14fcd907439ea37b7b64ce16b37d/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:12.118866 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62608a6d-4d15-4df1-8d80-377b19ebe667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4cbbf5be5e34be398d00dc3337b2e2b, user_id=e50b14fcd907439ea37b7b64ce16b37d, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:12.126632 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62608a6d-4d15-4df1-8d80-377b19ebe667 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:12.148502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1288/2605] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:56:12 2022] PUT /identity/v3/projects/b4cbbf5be5e34be398d00dc3337b2e2b/users/e50b14fcd907439ea37b7b64ce16b37d/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:56:12.167482 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc618d-04b5-4a0d-bc11-10760918634e None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:12.167951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc618d-04b5-4a0d-bc11-10760918634e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:12.168353 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc618d-04b5-4a0d-bc11-10760918634e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:12.205213 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-e7fc618d-04b5-4a0d-bc11-10760918634e None None] MFA Rules not processed for user `e50b14fcd907439ea37b7b64ce16b37d`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:56:12.206673 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e7fc618d-04b5-4a0d-bc11-10760918634e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:12.292930 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1318/2606] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:56:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:56:12.305524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62608a6d-4d15-4df1-8d80-377b19ebe667 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:12.307538 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-62608a6d-4d15-4df1-8d80-377b19ebe667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:12.390781 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62608a6d-4d15-4df1-8d80-377b19ebe667 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:12.398501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d936a-f35e-4f21-aef4-6ab358bd4c25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:12.399549 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d936a-f35e-4f21-aef4-6ab358bd4c25 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:12.400043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d936a-f35e-4f21-aef4-6ab358bd4c25 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:12.400534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d936a-f35e-4f21-aef4-6ab358bd4c25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:12.415623 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d936a-f35e-4f21-aef4-6ab358bd4c25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:12.418272 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d936a-f35e-4f21-aef4-6ab358bd4c25 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:12.433033 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1289/2607] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:56:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:56:20.923376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc618d-04b5-4a0d-bc11-10760918634e None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:20.939647 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:56:21.002340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc618d-04b5-4a0d-bc11-10760918634e None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:21.007671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805df954-a460-4c3e-b4b3-1f6378ea94b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:21.008603 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805df954-a460-4c3e-b4b3-1f6378ea94b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:21.008889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805df954-a460-4c3e-b4b3-1f6378ea94b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:21.009168 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805df954-a460-4c3e-b4b3-1f6378ea94b9 admin admin] PATH_INFO: `/v3/users/e50b14fcd907439ea37b7b64ce16b37d` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:21.010553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805df954-a460-4c3e-b4b3-1f6378ea94b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e50b14fcd907439ea37b7b64ce16b37d)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:21.012536 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805df954-a460-4c3e-b4b3-1f6378ea94b9 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:21.082321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-805df954-a460-4c3e-b4b3-1f6378ea94b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e50b14fcd907439ea37b7b64ce16b37d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:21.098208 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-805df954-a460-4c3e-b4b3-1f6378ea94b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e50b14fcd907439ea37b7b64ce16b37d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:21.098971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-805df954-a460-4c3e-b4b3-1f6378ea94b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e50b14fcd907439ea37b7b64ce16b37d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:21.110392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-805df954-a460-4c3e-b4b3-1f6378ea94b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e50b14fcd907439ea37b7b64ce16b37d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:21.124084 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-805df954-a460-4c3e-b4b3-1f6378ea94b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e50b14fcd907439ea37b7b64ce16b37d'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:21.134964 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1319/2608] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:56:20 2022] DELETE /identity/v3/users/e50b14fcd907439ea37b7b64ce16b37d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:56:21.140220 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a572d8d-3344-4d06-83f5-96094eaa6f5d None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:21.141105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a572d8d-3344-4d06-83f5-96094eaa6f5d None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:21.141105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a572d8d-3344-4d06-83f5-96094eaa6f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:21.160751 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-4a572d8d-3344-4d06-83f5-96094eaa6f5d None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:56:21.162335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4a572d8d-3344-4d06-83f5-96094eaa6f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:21.213304 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:56:21.238886 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1290/2609] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:56:21.251094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805df954-a460-4c3e-b4b3-1f6378ea94b9 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:21.252678 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-805df954-a460-4c3e-b4b3-1f6378ea94b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:21.325014 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805df954-a460-4c3e-b4b3-1f6378ea94b9 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:21.330505 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8402e-f1f0-4962-83cb-903d7ef9b2c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:21.331296 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8402e-f1f0-4962-83cb-903d7ef9b2c3 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:21.331626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8402e-f1f0-4962-83cb-903d7ef9b2c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:21.331972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8402e-f1f0-4962-83cb-903d7ef9b2c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:21.344446 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8402e-f1f0-4962-83cb-903d7ef9b2c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:21.346117 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8402e-f1f0-4962-83cb-903d7ef9b2c3 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:21.358321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1320/2610] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:56:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:56:21.739326 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a572d8d-3344-4d06-83f5-96094eaa6f5d None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:21.742470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4a572d8d-3344-4d06-83f5-96094eaa6f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:21.763557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a572d8d-3344-4d06-83f5-96094eaa6f5d None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:21.770594 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a9ea4-9e2e-41e1-ae94-1aae6011879d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:21.772370 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a9ea4-9e2e-41e1-ae94-1aae6011879d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:21.773669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a9ea4-9e2e-41e1-ae94-1aae6011879d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:21.774190 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a9ea4-9e2e-41e1-ae94-1aae6011879d admin admin] PATH_INFO: `/v3/projects/b4cbbf5be5e34be398d00dc3337b2e2b` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:21.776870 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a9ea4-9e2e-41e1-ae94-1aae6011879d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4cbbf5be5e34be398d00dc3337b2e2b)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:21.780880 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5a9ea4-9e2e-41e1-ae94-1aae6011879d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:21.851425 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ca5a9ea4-9e2e-41e1-ae94-1aae6011879d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4cbbf5be5e34be398d00dc3337b2e2b', 'request_id': 'req-ca5a9ea4-9e2e-41e1-ae94-1aae6011879d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:21.859723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ca5a9ea4-9e2e-41e1-ae94-1aae6011879d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4cbbf5be5e34be398d00dc3337b2e2b', 'request_id': 'req-ca5a9ea4-9e2e-41e1-ae94-1aae6011879d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:21.866914 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ca5a9ea4-9e2e-41e1-ae94-1aae6011879d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4cbbf5be5e34be398d00dc3337b2e2b', 'request_id': 'req-ca5a9ea4-9e2e-41e1-ae94-1aae6011879d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:21.882377 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ca5a9ea4-9e2e-41e1-ae94-1aae6011879d admin admin] The token cache is being invalidate because project b4cbbf5be5e34be398d00dc3337b2e2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:56:21.882676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ca5a9ea4-9e2e-41e1-ae94-1aae6011879d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:21.883367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ca5a9ea4-9e2e-41e1-ae94-1aae6011879d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:21.884977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1291/2611] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:56:21 2022] DELETE /identity/v3/projects/b4cbbf5be5e34be398d00dc3337b2e2b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:56:27.982955 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8402e-f1f0-4962-83cb-903d7ef9b2c3 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:27.994072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8402e-f1f0-4962-83cb-903d7ef9b2c3 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:28.034525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:56:28.062354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a2d3e-6c34-4e19-97ef-607bcbf0bdb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:28.063035 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a2d3e-6c34-4e19-97ef-607bcbf0bdb7 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:28.063320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a2d3e-6c34-4e19-97ef-607bcbf0bdb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:28.063602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a2d3e-6c34-4e19-97ef-607bcbf0bdb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:28.066145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f77a2d3e-6c34-4e19-97ef-607bcbf0bdb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:28.152440 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a2d3e-6c34-4e19-97ef-607bcbf0bdb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:28.154304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a2d3e-6c34-4e19-97ef-607bcbf0bdb7 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:28.168636 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1321/2612] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:56:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:56:31.878647 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:56:38.162482 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=35 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:56:46.298296 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a9ea4-9e2e-41e1-ae94-1aae6011879d admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:46.300031 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-ca5a9ea4-9e2e-41e1-ae94-1aae6011879d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:46.314662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a9ea4-9e2e-41e1-ae94-1aae6011879d admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:46.319133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10039073-5bae-4dfe-bb99-e018b100cd4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:46.319808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10039073-5bae-4dfe-bb99-e018b100cd4f service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:46.320097 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10039073-5bae-4dfe-bb99-e018b100cd4f service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:46.320373 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10039073-5bae-4dfe-bb99-e018b100cd4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:46.322686 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-10039073-5bae-4dfe-bb99-e018b100cd4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:46.415642 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10039073-5bae-4dfe-bb99-e018b100cd4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:46.417546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10039073-5bae-4dfe-bb99-e018b100cd4f service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:46.429457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1292/2613] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:56:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:56:51.383721 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a2d3e-6c34-4e19-97ef-607bcbf0bdb7 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:51.386193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f77a2d3e-6c34-4e19-97ef-607bcbf0bdb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:51.407078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a2d3e-6c34-4e19-97ef-607bcbf0bdb7 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:51.412682 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e1da5-87e8-4622-8957-cc648dc56d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:51.413807 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e1da5-87e8-4622-8957-cc648dc56d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:51.414197 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e1da5-87e8-4622-8957-cc648dc56d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:51.414540 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e1da5-87e8-4622-8957-cc648dc56d50 admin admin] PATH_INFO: `/v3/users/6890639a0a2e4ca9b103919fdbbffc5b` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:51.415956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e1da5-87e8-4622-8957-cc648dc56d50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6890639a0a2e4ca9b103919fdbbffc5b)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:51.418274 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e1da5-87e8-4622-8957-cc648dc56d50 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:51.492363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4e9e1da5-87e8-4622-8957-cc648dc56d50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6890639a0a2e4ca9b103919fdbbffc5b'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:51.508189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4e9e1da5-87e8-4622-8957-cc648dc56d50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6890639a0a2e4ca9b103919fdbbffc5b'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:51.509217 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4e9e1da5-87e8-4622-8957-cc648dc56d50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6890639a0a2e4ca9b103919fdbbffc5b'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:51.520138 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4e9e1da5-87e8-4622-8957-cc648dc56d50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6890639a0a2e4ca9b103919fdbbffc5b'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:51.530773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4e9e1da5-87e8-4622-8957-cc648dc56d50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6890639a0a2e4ca9b103919fdbbffc5b'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:51.539490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1322/2614] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:56:51 2022] DELETE /identity/v3/users/6890639a0a2e4ca9b103919fdbbffc5b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:56:51.826184 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10039073-5bae-4dfe-bb99-e018b100cd4f service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:51.845411 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10039073-5bae-4dfe-bb99-e018b100cd4f service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:51.933893 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee22e0-fd62-4673-b8fb-257eb0cf0260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:51.934798 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee22e0-fd62-4673-b8fb-257eb0cf0260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:51.935162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee22e0-fd62-4673-b8fb-257eb0cf0260 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:51.935347 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee22e0-fd62-4673-b8fb-257eb0cf0260 admin admin] PATH_INFO: `/v3/projects/5d416620e9f942ab9f76e6b117cd8b61` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:51.936728 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee22e0-fd62-4673-b8fb-257eb0cf0260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d416620e9f942ab9f76e6b117cd8b61)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:51.938755 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee22e0-fd62-4673-b8fb-257eb0cf0260 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:52.004707 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-72ee22e0-fd62-4673-b8fb-257eb0cf0260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d416620e9f942ab9f76e6b117cd8b61', 'request_id': 'req-72ee22e0-fd62-4673-b8fb-257eb0cf0260'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:52.018881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-72ee22e0-fd62-4673-b8fb-257eb0cf0260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d416620e9f942ab9f76e6b117cd8b61', 'request_id': 'req-72ee22e0-fd62-4673-b8fb-257eb0cf0260'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:52.027768 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-72ee22e0-fd62-4673-b8fb-257eb0cf0260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d416620e9f942ab9f76e6b117cd8b61', 'request_id': 'req-72ee22e0-fd62-4673-b8fb-257eb0cf0260'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:52.047657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-72ee22e0-fd62-4673-b8fb-257eb0cf0260 admin admin] The token cache is being invalidate because project 5d416620e9f942ab9f76e6b117cd8b61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:56:52.048081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-72ee22e0-fd62-4673-b8fb-257eb0cf0260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:52.049149 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-72ee22e0-fd62-4673-b8fb-257eb0cf0260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:52.051699 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1293/2615] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:56:51 2022] DELETE /identity/v3/projects/5d416620e9f942ab9f76e6b117cd8b61 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:56:52.059541 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b8410-1eb7-48fe-8749-2cccb0af0431 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:52.059997 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b8410-1eb7-48fe-8749-2cccb0af0431 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:52.060396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b8410-1eb7-48fe-8749-2cccb0af0431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:52.085357 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-d04b8410-1eb7-48fe-8749-2cccb0af0431 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:56:52.086843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-d04b8410-1eb7-48fe-8749-2cccb0af0431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:52.197302 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1323/2616] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:56:52.199507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff2d8f-356e-4b88-91eb-d00ffd0cacc1 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:52.199507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff2d8f-356e-4b88-91eb-d00ffd0cacc1 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:52.199831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff2d8f-356e-4b88-91eb-d00ffd0cacc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:52.223069 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-32ff2d8f-356e-4b88-91eb-d00ffd0cacc1 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:56:52.225131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-32ff2d8f-356e-4b88-91eb-d00ffd0cacc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:52.234455 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1294/2617] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:56:52.239716 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b8410-1eb7-48fe-8749-2cccb0af0431 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:52.256686 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b8410-1eb7-48fe-8749-2cccb0af0431 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:52.264785 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f42f14-0236-4b9e-a186-ea815927a441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:52.266329 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f42f14-0236-4b9e-a186-ea815927a441 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:52.266860 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f42f14-0236-4b9e-a186-ea815927a441 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:52.267845 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f42f14-0236-4b9e-a186-ea815927a441 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:52.268891 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f42f14-0236-4b9e-a186-ea815927a441 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:56:52.269501 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f42f14-0236-4b9e-a186-ea815927a441 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:52.273368 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f42f14-0236-4b9e-a186-ea815927a441 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:52.299196 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1324/2618] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:56:52.305824 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff2d8f-356e-4b88-91eb-d00ffd0cacc1 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:52.324086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff2d8f-356e-4b88-91eb-d00ffd0cacc1 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:52.333875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e270f8-f4cc-4fc1-9cbd-1aa93cf6674e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:52.339320 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e270f8-f4cc-4fc1-9cbd-1aa93cf6674e admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:52.339835 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e270f8-f4cc-4fc1-9cbd-1aa93cf6674e admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:52.340521 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e270f8-f4cc-4fc1-9cbd-1aa93cf6674e admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:52.341887 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e270f8-f4cc-4fc1-9cbd-1aa93cf6674e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:52.345055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e270f8-f4cc-4fc1-9cbd-1aa93cf6674e admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:52.369329 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:56:52.393780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1295/2619] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:56:52 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:56:52.395139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f42f14-0236-4b9e-a186-ea815927a441 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:52.408296 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f42f14-0236-4b9e-a186-ea815927a441 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:52.493765 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df667c5d-229a-4fe1-978e-f06bc7b62375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:52.494395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df667c5d-229a-4fe1-978e-f06bc7b62375 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:52.494679 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df667c5d-229a-4fe1-978e-f06bc7b62375 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:52.494959 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df667c5d-229a-4fe1-978e-f06bc7b62375 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:52.495693 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df667c5d-229a-4fe1-978e-f06bc7b62375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:52.497778 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df667c5d-229a-4fe1-978e-f06bc7b62375 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:52.506016 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:56:52.538909 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1325/2620] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:56:52 2022] POST /identity/v3/users => generated 286 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:56:52.543652 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e270f8-f4cc-4fc1-9cbd-1aa93cf6674e admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:52.561945 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e270f8-f4cc-4fc1-9cbd-1aa93cf6674e admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:52.567201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7244cc5-2d58-4dd6-911f-02e81b5999de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:52.568453 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7244cc5-2d58-4dd6-911f-02e81b5999de admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:52.568797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7244cc5-2d58-4dd6-911f-02e81b5999de admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:52.569090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7244cc5-2d58-4dd6-911f-02e81b5999de admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:52.569917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7244cc5-2d58-4dd6-911f-02e81b5999de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:52.571642 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7244cc5-2d58-4dd6-911f-02e81b5999de admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:52.589770 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1296/2621] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:56:52.591247 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df667c5d-229a-4fe1-978e-f06bc7b62375 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:52.605066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df667c5d-229a-4fe1-978e-f06bc7b62375 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:52.610352 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cb603-e31d-4643-8a13-969cfb3a07ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:52.611093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cb603-e31d-4643-8a13-969cfb3a07ed admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:52.611376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cb603-e31d-4643-8a13-969cfb3a07ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:52.611651 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cb603-e31d-4643-8a13-969cfb3a07ed admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:52.612411 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9cb603-e31d-4643-8a13-969cfb3a07ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:52.614696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9cb603-e31d-4643-8a13-969cfb3a07ed admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:52.627461 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1326/2622] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:56:52.633356 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7244cc5-2d58-4dd6-911f-02e81b5999de admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:52.645677 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7244cc5-2d58-4dd6-911f-02e81b5999de admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:52.650874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03a6aa-da90-48ef-bec9-e80564e059ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:52.651830 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03a6aa-da90-48ef-bec9-e80564e059ac admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:52.652110 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03a6aa-da90-48ef-bec9-e80564e059ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:52.652574 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03a6aa-da90-48ef-bec9-e80564e059ac admin admin] PATH_INFO: `/v3/projects/854bcd38c49a4a6b8123b58d7deae9d3/users/e7e01930895646ee94ab7e30ad2e49d0/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:52.671308 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03a6aa-da90-48ef-bec9-e80564e059ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=854bcd38c49a4a6b8123b58d7deae9d3, user_id=e7e01930895646ee94ab7e30ad2e49d0, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:52.674193 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03a6aa-da90-48ef-bec9-e80564e059ac admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:52.691749 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1297/2623] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:56:52 2022] PUT /identity/v3/projects/854bcd38c49a4a6b8123b58d7deae9d3/users/e7e01930895646ee94ab7e30ad2e49d0/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:56:52.704206 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b378f9e-8e9f-4439-bc71-cc2722bae4cb None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:52.704709 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b378f9e-8e9f-4439-bc71-cc2722bae4cb None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:52.705208 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b378f9e-8e9f-4439-bc71-cc2722bae4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:52.748515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-3b378f9e-8e9f-4439-bc71-cc2722bae4cb None None] MFA Rules not processed for user `e7e01930895646ee94ab7e30ad2e49d0`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:56:52.750078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-3b378f9e-8e9f-4439-bc71-cc2722bae4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:52.864289 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1327/2624] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:56:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:56:52.882374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03a6aa-da90-48ef-bec9-e80564e059ac admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:52.884986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-9d03a6aa-da90-48ef-bec9-e80564e059ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:52.987319 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03a6aa-da90-48ef-bec9-e80564e059ac admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:52.998361 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290756ae-6590-4945-b31e-3edcc5d8a4ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:52.999552 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290756ae-6590-4945-b31e-3edcc5d8a4ba service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:53.000003 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290756ae-6590-4945-b31e-3edcc5d8a4ba service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:53.000528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290756ae-6590-4945-b31e-3edcc5d8a4ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:53.018165 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290756ae-6590-4945-b31e-3edcc5d8a4ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:53.022378 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290756ae-6590-4945-b31e-3edcc5d8a4ba service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:53.040921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1298/2625] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:56:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:56:53.758190 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e478c-5031-4928-aae4-26e8c4f81169 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:53.759039 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e478c-5031-4928-aae4-26e8c4f81169 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:53.759039 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e478c-5031-4928-aae4-26e8c4f81169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:53.778102 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-f34e478c-5031-4928-aae4-26e8c4f81169 None None] MFA Rules not processed for user `ce59e0072f1c4b91a2825e5fecde7e23`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:56:53.779381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f34e478c-5031-4928-aae4-26e8c4f81169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:53.849370 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1328/2626] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:56:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:56:56.017028 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290756ae-6590-4945-b31e-3edcc5d8a4ba service nova] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:56.106477 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290756ae-6590-4945-b31e-3edcc5d8a4ba service nova] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:56.111329 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cea112-f052-47a4-99f0-6ec49bf23c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:56.112317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cea112-f052-47a4-99f0-6ec49bf23c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:56.112652 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cea112-f052-47a4-99f0-6ec49bf23c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:56.113029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cea112-f052-47a4-99f0-6ec49bf23c1c admin admin] PATH_INFO: `/v3/users/e7e01930895646ee94ab7e30ad2e49d0` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:56.114421 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cea112-f052-47a4-99f0-6ec49bf23c1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e01930895646ee94ab7e30ad2e49d0)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:56.116406 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cea112-f052-47a4-99f0-6ec49bf23c1c admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:56.176622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e9cea112-f052-47a4-99f0-6ec49bf23c1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e01930895646ee94ab7e30ad2e49d0'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:56.178463 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:56:56.205170 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e9cea112-f052-47a4-99f0-6ec49bf23c1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e01930895646ee94ab7e30ad2e49d0'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:56.220567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e9cea112-f052-47a4-99f0-6ec49bf23c1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e01930895646ee94ab7e30ad2e49d0'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:56.221344 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e9cea112-f052-47a4-99f0-6ec49bf23c1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e01930895646ee94ab7e30ad2e49d0'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:56.222182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e9cea112-f052-47a4-99f0-6ec49bf23c1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e01930895646ee94ab7e30ad2e49d0'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:56.244694 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1299/2627] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:56:56 2022] DELETE /identity/v3/users/e7e01930895646ee94ab7e30ad2e49d0 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:56:56.250962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf852d7f-9c49-4ed4-aeed-4f10cbb549cf None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:56.251635 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf852d7f-9c49-4ed4-aeed-4f10cbb549cf None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:56.251981 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf852d7f-9c49-4ed4-aeed-4f10cbb549cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:56.287153 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-bf852d7f-9c49-4ed4-aeed-4f10cbb549cf None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:56:56.289147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-bf852d7f-9c49-4ed4-aeed-4f10cbb549cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:56.372384 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1329/2628] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:56:56.384375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cea112-f052-47a4-99f0-6ec49bf23c1c admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:56.386671 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e9cea112-f052-47a4-99f0-6ec49bf23c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:56.470765 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cea112-f052-47a4-99f0-6ec49bf23c1c admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:56.475566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9f358-5b75-45fe-9849-31529a1bc8f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:56.476265 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9f358-5b75-45fe-9849-31529a1bc8f7 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:56.476567 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9f358-5b75-45fe-9849-31529a1bc8f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:56.476878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9f358-5b75-45fe-9849-31529a1bc8f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:56.493491 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9f358-5b75-45fe-9849-31529a1bc8f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:56.495033 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9f358-5b75-45fe-9849-31529a1bc8f7 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:56.506921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1300/2629] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:56:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:56:56.904756 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf852d7f-9c49-4ed4-aeed-4f10cbb549cf None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:56.907195 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-bf852d7f-9c49-4ed4-aeed-4f10cbb549cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:56.928044 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf852d7f-9c49-4ed4-aeed-4f10cbb549cf None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:56.947702 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833fc62-94f0-4389-87ff-9bd6c89f8e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:56.949753 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833fc62-94f0-4389-87ff-9bd6c89f8e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:56.950844 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833fc62-94f0-4389-87ff-9bd6c89f8e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:56.952072 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833fc62-94f0-4389-87ff-9bd6c89f8e6a admin admin] PATH_INFO: `/v3/projects/854bcd38c49a4a6b8123b58d7deae9d3` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:56.956646 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833fc62-94f0-4389-87ff-9bd6c89f8e6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=854bcd38c49a4a6b8123b58d7deae9d3)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:56.961795 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833fc62-94f0-4389-87ff-9bd6c89f8e6a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:56.984947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:56:57.059789 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-1833fc62-94f0-4389-87ff-9bd6c89f8e6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '854bcd38c49a4a6b8123b58d7deae9d3', 'request_id': 'req-1833fc62-94f0-4389-87ff-9bd6c89f8e6a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:57.068013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-1833fc62-94f0-4389-87ff-9bd6c89f8e6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '854bcd38c49a4a6b8123b58d7deae9d3', 'request_id': 'req-1833fc62-94f0-4389-87ff-9bd6c89f8e6a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:57.078396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-1833fc62-94f0-4389-87ff-9bd6c89f8e6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '854bcd38c49a4a6b8123b58d7deae9d3', 'request_id': 'req-1833fc62-94f0-4389-87ff-9bd6c89f8e6a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:57.094565 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-1833fc62-94f0-4389-87ff-9bd6c89f8e6a admin admin] The token cache is being invalidate because project 854bcd38c49a4a6b8123b58d7deae9d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:56:57.094925 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-1833fc62-94f0-4389-87ff-9bd6c89f8e6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:57.095679 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-1833fc62-94f0-4389-87ff-9bd6c89f8e6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:57.097522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1330/2630] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:56:56 2022] DELETE /identity/v3/projects/854bcd38c49a4a6b8123b58d7deae9d3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:56:57.106654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb99c1e-91ff-4eab-acaa-5954749d6dec None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:57.107213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb99c1e-91ff-4eab-acaa-5954749d6dec None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:57.107686 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb99c1e-91ff-4eab-acaa-5954749d6dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:57.146201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-7bb99c1e-91ff-4eab-acaa-5954749d6dec None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:56:57.149402 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-7bb99c1e-91ff-4eab-acaa-5954749d6dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:57.253789 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1301/2631] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:56:57.269095 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718f1b8-061d-484a-a614-5a0c957dc98f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:57.269783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718f1b8-061d-484a-a614-5a0c957dc98f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:57.269783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718f1b8-061d-484a-a614-5a0c957dc98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:57.298829 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-5718f1b8-061d-484a-a614-5a0c957dc98f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:56:57.300214 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-5718f1b8-061d-484a-a614-5a0c957dc98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:57.311874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1331/2632] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:56:57.316259 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb99c1e-91ff-4eab-acaa-5954749d6dec None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:57.327991 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb99c1e-91ff-4eab-acaa-5954749d6dec None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:57.333860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad8b76-7154-4ba7-82e3-759c1bd6bbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:57.334198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad8b76-7154-4ba7-82e3-759c1bd6bbbd admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:57.334376 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad8b76-7154-4ba7-82e3-759c1bd6bbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:57.334655 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad8b76-7154-4ba7-82e3-759c1bd6bbbd admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:57.335551 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad8b76-7154-4ba7-82e3-759c1bd6bbbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:56:57.335957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad8b76-7154-4ba7-82e3-759c1bd6bbbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:57.337621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad8b76-7154-4ba7-82e3-759c1bd6bbbd admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:57.354060 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1302/2633] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:56:57.357958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718f1b8-061d-484a-a614-5a0c957dc98f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:57.374433 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718f1b8-061d-484a-a614-5a0c957dc98f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:57.379200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd93708-83db-4e66-bf9e-958e2a271d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:57.380408 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd93708-83db-4e66-bf9e-958e2a271d91 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:57.380751 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd93708-83db-4e66-bf9e-958e2a271d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:57.381050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd93708-83db-4e66-bf9e-958e2a271d91 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:57.382153 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd93708-83db-4e66-bf9e-958e2a271d91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:57.384165 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd93708-83db-4e66-bf9e-958e2a271d91 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:57.425936 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1332/2634] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:56:57 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:56:57.432489 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad8b76-7154-4ba7-82e3-759c1bd6bbbd admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:57.449630 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad8b76-7154-4ba7-82e3-759c1bd6bbbd admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:57.549062 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:56:57.565126 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58845232-d700-43e7-bca4-8a65d3cf2969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:57.566421 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58845232-d700-43e7-bca4-8a65d3cf2969 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:57.567921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58845232-d700-43e7-bca4-8a65d3cf2969 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:57.568391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58845232-d700-43e7-bca4-8a65d3cf2969 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:57.569824 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58845232-d700-43e7-bca4-8a65d3cf2969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:57.574659 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58845232-d700-43e7-bca4-8a65d3cf2969 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:57.628880 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1303/2635] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:56:57 2022] POST /identity/v3/users => generated 288 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:56:57.635674 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd93708-83db-4e66-bf9e-958e2a271d91 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:57.653731 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd93708-83db-4e66-bf9e-958e2a271d91 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:57.659513 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99256be2-005b-4905-95c7-611f14c08e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:57.660587 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99256be2-005b-4905-95c7-611f14c08e5f admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:57.662169 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99256be2-005b-4905-95c7-611f14c08e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:57.663939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99256be2-005b-4905-95c7-611f14c08e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:57.665598 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99256be2-005b-4905-95c7-611f14c08e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:57.668328 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99256be2-005b-4905-95c7-611f14c08e5f admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:57.693375 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1333/2636] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:56:57.698863 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58845232-d700-43e7-bca4-8a65d3cf2969 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:57.713477 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58845232-d700-43e7-bca4-8a65d3cf2969 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:57.720124 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62fc663-2f3f-40f2-9a2b-784eaa38c4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:57.720976 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fc663-2f3f-40f2-9a2b-784eaa38c4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:57.721532 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fc663-2f3f-40f2-9a2b-784eaa38c4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:57.722149 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fc663-2f3f-40f2-9a2b-784eaa38c4a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:57.723095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62fc663-2f3f-40f2-9a2b-784eaa38c4a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:57.726236 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62fc663-2f3f-40f2-9a2b-784eaa38c4a8 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:57.739488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1304/2637] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:56:57.746939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99256be2-005b-4905-95c7-611f14c08e5f admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:57.758922 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99256be2-005b-4905-95c7-611f14c08e5f admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:57.765133 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03304de2-441c-4c98-b475-1fca090510ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:57.766432 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03304de2-441c-4c98-b475-1fca090510ec admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:57.766823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03304de2-441c-4c98-b475-1fca090510ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:57.767220 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03304de2-441c-4c98-b475-1fca090510ec admin admin] PATH_INFO: `/v3/projects/990fc8cfabf847538fa9ff2b01932c44/users/11d2dd62463b4a709615089d3e534eb4/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:57.789460 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03304de2-441c-4c98-b475-1fca090510ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=990fc8cfabf847538fa9ff2b01932c44, user_id=11d2dd62463b4a709615089d3e534eb4, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:57.793044 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03304de2-441c-4c98-b475-1fca090510ec admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:57.836470 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1334/2638] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:56:57 2022] PUT /identity/v3/projects/990fc8cfabf847538fa9ff2b01932c44/users/11d2dd62463b4a709615089d3e534eb4/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:56:57.853474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a18db-ae9c-4a4c-aaa3-d220c88e6a73 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:57.853851 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a18db-ae9c-4a4c-aaa3-d220c88e6a73 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:57.854349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a18db-ae9c-4a4c-aaa3-d220c88e6a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:57.901697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-bf0a18db-ae9c-4a4c-aaa3-d220c88e6a73 None None] MFA Rules not processed for user `11d2dd62463b4a709615089d3e534eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:56:57.903888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-bf0a18db-ae9c-4a4c-aaa3-d220c88e6a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:58.027461 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1305/2639] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:56:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:56:58.044413 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03304de2-441c-4c98-b475-1fca090510ec admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:58.046558 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-03304de2-441c-4c98-b475-1fca090510ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:58.187518 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03304de2-441c-4c98-b475-1fca090510ec admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:58.198047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa5f3f-0ab4-4eb7-beb2-11ec8c27a14d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:58.199101 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa5f3f-0ab4-4eb7-beb2-11ec8c27a14d service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:58.199610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa5f3f-0ab4-4eb7-beb2-11ec8c27a14d service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:58.200118 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa5f3f-0ab4-4eb7-beb2-11ec8c27a14d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:58.217042 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa5f3f-0ab4-4eb7-beb2-11ec8c27a14d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:58.220190 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa5f3f-0ab4-4eb7-beb2-11ec8c27a14d service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:58.237864 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1335/2640] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:56:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:56:58.554795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a18db-ae9c-4a4c-aaa3-d220c88e6a73 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:58.638894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a18db-ae9c-4a4c-aaa3-d220c88e6a73 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:58.643610 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0d53a-97f6-4ab1-9711-60e308b8f719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:58.644509 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0d53a-97f6-4ab1-9711-60e308b8f719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:58.644793 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0d53a-97f6-4ab1-9711-60e308b8f719 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:58.645068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0d53a-97f6-4ab1-9711-60e308b8f719 admin admin] PATH_INFO: `/v3/users/11d2dd62463b4a709615089d3e534eb4` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:58.646874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0d53a-97f6-4ab1-9711-60e308b8f719 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11d2dd62463b4a709615089d3e534eb4)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:58.648856 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0d53a-97f6-4ab1-9711-60e308b8f719 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:58.722632 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-96f0d53a-97f6-4ab1-9711-60e308b8f719 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11d2dd62463b4a709615089d3e534eb4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:58.742450 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-96f0d53a-97f6-4ab1-9711-60e308b8f719 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11d2dd62463b4a709615089d3e534eb4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:58.744660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:56:58.754974 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-96f0d53a-97f6-4ab1-9711-60e308b8f719 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11d2dd62463b4a709615089d3e534eb4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:58.756313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-96f0d53a-97f6-4ab1-9711-60e308b8f719 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11d2dd62463b4a709615089d3e534eb4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:58.758021 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-96f0d53a-97f6-4ab1-9711-60e308b8f719 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11d2dd62463b4a709615089d3e534eb4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:58.781104 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1306/2641] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:56:58 2022] DELETE /identity/v3/users/11d2dd62463b4a709615089d3e534eb4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:56:58.787202 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062acc5-2b67-4e9e-a606-39cf82dabf9f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:58.788060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062acc5-2b67-4e9e-a606-39cf82dabf9f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:58.788235 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062acc5-2b67-4e9e-a606-39cf82dabf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:58.817813 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-7062acc5-2b67-4e9e-a606-39cf82dabf9f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:56:58.819820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-7062acc5-2b67-4e9e-a606-39cf82dabf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:58.836781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:56:58.910227 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1336/2642] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:56:58.922079 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0d53a-97f6-4ab1-9711-60e308b8f719 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:58.923572 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-96f0d53a-97f6-4ab1-9711-60e308b8f719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:59.002861 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0d53a-97f6-4ab1-9711-60e308b8f719 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:59.006751 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8134b5-56fc-462a-ac06-07b2fb66b637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:59.007444 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8134b5-56fc-462a-ac06-07b2fb66b637 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:59.007761 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8134b5-56fc-462a-ac06-07b2fb66b637 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:59.008114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8134b5-56fc-462a-ac06-07b2fb66b637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:59.018791 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8134b5-56fc-462a-ac06-07b2fb66b637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:59.020416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8134b5-56fc-462a-ac06-07b2fb66b637 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:59.031905 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1307/2643] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:56:59.361208 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062acc5-2b67-4e9e-a606-39cf82dabf9f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:59.362703 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-7062acc5-2b67-4e9e-a606-39cf82dabf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:59.381319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062acc5-2b67-4e9e-a606-39cf82dabf9f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:59.388688 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab892c73-5b58-4d7c-8c1f-bc594ccf3d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:59.389916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab892c73-5b58-4d7c-8c1f-bc594ccf3d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:59.390297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab892c73-5b58-4d7c-8c1f-bc594ccf3d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:59.390632 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab892c73-5b58-4d7c-8c1f-bc594ccf3d9a admin admin] PATH_INFO: `/v3/projects/990fc8cfabf847538fa9ff2b01932c44` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:59.392544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab892c73-5b58-4d7c-8c1f-bc594ccf3d9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=990fc8cfabf847538fa9ff2b01932c44)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:59.395138 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab892c73-5b58-4d7c-8c1f-bc594ccf3d9a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:59.457109 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ab892c73-5b58-4d7c-8c1f-bc594ccf3d9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '990fc8cfabf847538fa9ff2b01932c44', 'request_id': 'req-ab892c73-5b58-4d7c-8c1f-bc594ccf3d9a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:59.464259 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ab892c73-5b58-4d7c-8c1f-bc594ccf3d9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '990fc8cfabf847538fa9ff2b01932c44', 'request_id': 'req-ab892c73-5b58-4d7c-8c1f-bc594ccf3d9a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:59.473367 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ab892c73-5b58-4d7c-8c1f-bc594ccf3d9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '990fc8cfabf847538fa9ff2b01932c44', 'request_id': 'req-ab892c73-5b58-4d7c-8c1f-bc594ccf3d9a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:59.485619 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ab892c73-5b58-4d7c-8c1f-bc594ccf3d9a admin admin] The token cache is being invalidate because project 990fc8cfabf847538fa9ff2b01932c44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:56:59.485951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ab892c73-5b58-4d7c-8c1f-bc594ccf3d9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:59.486609 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ab892c73-5b58-4d7c-8c1f-bc594ccf3d9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:56:59.488200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1337/2644] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:56:59 2022] DELETE /identity/v3/projects/990fc8cfabf847538fa9ff2b01932c44 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:56:59.495036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddeb96-2cd4-4a48-b2fb-6749da0e933a None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:59.495295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddeb96-2cd4-4a48-b2fb-6749da0e933a None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:59.495578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddeb96-2cd4-4a48-b2fb-6749da0e933a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:59.516052 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-3fddeb96-2cd4-4a48-b2fb-6749da0e933a None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:56:59.517315 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3fddeb96-2cd4-4a48-b2fb-6749da0e933a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:59.591397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1308/2645] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:56:59.603741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2418c50-3767-4f91-9693-1fc0e2059d2b None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:59.604215 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2418c50-3767-4f91-9693-1fc0e2059d2b None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:59.604608 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2418c50-3767-4f91-9693-1fc0e2059d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:59.627734 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-b2418c50-3767-4f91-9693-1fc0e2059d2b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:56:59.628983 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-b2418c50-3767-4f91-9693-1fc0e2059d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:56:59.638486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1338/2646] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:56:59.643083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddeb96-2cd4-4a48-b2fb-6749da0e933a None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:59.656020 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddeb96-2cd4-4a48-b2fb-6749da0e933a None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:59.660930 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0685fc0-bc51-45fe-b1af-dc46d8b2233e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:59.661612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0685fc0-bc51-45fe-b1af-dc46d8b2233e admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:59.661957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0685fc0-bc51-45fe-b1af-dc46d8b2233e admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:59.662235 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0685fc0-bc51-45fe-b1af-dc46d8b2233e admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:59.662985 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0685fc0-bc51-45fe-b1af-dc46d8b2233e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:56:59.663301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0685fc0-bc51-45fe-b1af-dc46d8b2233e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:59.664976 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0685fc0-bc51-45fe-b1af-dc46d8b2233e admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:59.680298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1309/2647] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:56:59.684918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2418c50-3767-4f91-9693-1fc0e2059d2b None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:59.701555 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2418c50-3767-4f91-9693-1fc0e2059d2b None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:59.709295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f544afa-99d4-4964-a14f-13922b27d7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:59.710238 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f544afa-99d4-4964-a14f-13922b27d7ae admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:59.710768 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f544afa-99d4-4964-a14f-13922b27d7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:59.711133 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f544afa-99d4-4964-a14f-13922b27d7ae admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:59.711968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f544afa-99d4-4964-a14f-13922b27d7ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:59.714595 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f544afa-99d4-4964-a14f-13922b27d7ae admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:59.752961 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1339/2648] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:56:59 2022] POST /identity/v3/projects => generated 409 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:56:59.758645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0685fc0-bc51-45fe-b1af-dc46d8b2233e admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:59.770131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0685fc0-bc51-45fe-b1af-dc46d8b2233e admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:59.845309 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8149a-066d-4575-bc6f-59a21f7a5b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:59.846075 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8149a-066d-4575-bc6f-59a21f7a5b38 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:59.846359 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8149a-066d-4575-bc6f-59a21f7a5b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:59.846639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8149a-066d-4575-bc6f-59a21f7a5b38 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:59.847365 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8149a-066d-4575-bc6f-59a21f7a5b38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:59.850640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8149a-066d-4575-bc6f-59a21f7a5b38 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:59.880233 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1310/2649] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:56:59 2022] POST /identity/v3/users => generated 305 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:56:59.885467 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f544afa-99d4-4964-a14f-13922b27d7ae admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:59.904555 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f544afa-99d4-4964-a14f-13922b27d7ae admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:59.912798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700df1f5-88d0-41e5-b911-8f111414a185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:59.913522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700df1f5-88d0-41e5-b911-8f111414a185 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:59.913849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700df1f5-88d0-41e5-b911-8f111414a185 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:59.914129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700df1f5-88d0-41e5-b911-8f111414a185 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:59.914891 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700df1f5-88d0-41e5-b911-8f111414a185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:59.916548 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700df1f5-88d0-41e5-b911-8f111414a185 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:59.929853 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1340/2650] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:56:59.934609 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8149a-066d-4575-bc6f-59a21f7a5b38 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:56:59.940399 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:56:59.949811 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8149a-066d-4575-bc6f-59a21f7a5b38 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:56:59.958304 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b53d64-58d4-44c9-bb84-1dd9bf76aecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:56:59.958922 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b53d64-58d4-44c9-bb84-1dd9bf76aecd admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:56:59.959217 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b53d64-58d4-44c9-bb84-1dd9bf76aecd admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:56:59.959501 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b53d64-58d4-44c9-bb84-1dd9bf76aecd admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:56:59.960259 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b53d64-58d4-44c9-bb84-1dd9bf76aecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:56:59.961989 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b53d64-58d4-44c9-bb84-1dd9bf76aecd admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:56:59.974531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1311/2651] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:56:59.981232 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700df1f5-88d0-41e5-b911-8f111414a185 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:00.000705 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700df1f5-88d0-41e5-b911-8f111414a185 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:00.009508 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bf1d5-96b2-4b41-9805-2bceda8be904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:00.011079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bf1d5-96b2-4b41-9805-2bceda8be904 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:00.011552 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bf1d5-96b2-4b41-9805-2bceda8be904 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:00.012034 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bf1d5-96b2-4b41-9805-2bceda8be904 admin admin] PATH_INFO: `/v3/projects/2c41c1d9807e4d04a10148f55f2486c1/users/7689026a1ac44b9aaa1dcfec0f79b0b4/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:00.041398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9bf1d5-96b2-4b41-9805-2bceda8be904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c41c1d9807e4d04a10148f55f2486c1, user_id=7689026a1ac44b9aaa1dcfec0f79b0b4, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:00.047240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9bf1d5-96b2-4b41-9805-2bceda8be904 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:00.069244 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1341/2652] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:56:59 2022] PUT /identity/v3/projects/2c41c1d9807e4d04a10148f55f2486c1/users/7689026a1ac44b9aaa1dcfec0f79b0b4/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:00.081855 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8eed9-0bae-41e3-a5f4-430a660987ea None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:00.082158 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8eed9-0bae-41e3-a5f4-430a660987ea None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:00.082421 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8eed9-0bae-41e3-a5f4-430a660987ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:00.122797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-eba8eed9-0bae-41e3-a5f4-430a660987ea None None] MFA Rules not processed for user `7689026a1ac44b9aaa1dcfec0f79b0b4`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:57:00.124440 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-eba8eed9-0bae-41e3-a5f4-430a660987ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:00.200168 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1312/2653] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:57:00 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:57:00.205029 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bf1d5-96b2-4b41-9805-2bceda8be904 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:00.252578 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:00.284982 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bf1d5-96b2-4b41-9805-2bceda8be904 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:00.291752 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453fe38-b16a-40d6-b53a-ec6c52d66280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:00.292432 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453fe38-b16a-40d6-b53a-ec6c52d66280 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:00.292716 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453fe38-b16a-40d6-b53a-ec6c52d66280 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:00.293001 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453fe38-b16a-40d6-b53a-ec6c52d66280 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:00.293735 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453fe38-b16a-40d6-b53a-ec6c52d66280 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:00.295937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453fe38-b16a-40d6-b53a-ec6c52d66280 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:00.325115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1342/2654] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:57:00 2022] POST /identity/v3/projects => generated 409 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:57:00.329976 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8eed9-0bae-41e3-a5f4-430a660987ea None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:00.342877 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8eed9-0bae-41e3-a5f4-430a660987ea None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:00.407528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c6a63-3f4e-4e99-b27b-1afd5f7efd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:00.408232 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c6a63-3f4e-4e99-b27b-1afd5f7efd47 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:00.408546 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c6a63-3f4e-4e99-b27b-1afd5f7efd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:00.408793 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c6a63-3f4e-4e99-b27b-1afd5f7efd47 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:00.409509 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c6a63-3f4e-4e99-b27b-1afd5f7efd47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:00.411770 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c6a63-3f4e-4e99-b27b-1afd5f7efd47 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:00.435810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1313/2655] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:57:00 2022] POST /identity/v3/users => generated 311 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:57:00.442671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453fe38-b16a-40d6-b53a-ec6c52d66280 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:00.454654 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453fe38-b16a-40d6-b53a-ec6c52d66280 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:00.460449 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b634f7-c8fd-41be-85fe-b100a4dc907d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:00.461191 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b634f7-c8fd-41be-85fe-b100a4dc907d admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:00.461577 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b634f7-c8fd-41be-85fe-b100a4dc907d admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:00.461952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b634f7-c8fd-41be-85fe-b100a4dc907d admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:00.462746 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b634f7-c8fd-41be-85fe-b100a4dc907d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:00.464457 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b634f7-c8fd-41be-85fe-b100a4dc907d admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:00.477027 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1343/2656] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:57:00.481861 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c6a63-3f4e-4e99-b27b-1afd5f7efd47 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:00.495908 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c6a63-3f4e-4e99-b27b-1afd5f7efd47 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:00.504400 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b96a4-2602-4b9a-99b3-93380dc0dac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:00.506654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b96a4-2602-4b9a-99b3-93380dc0dac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:00.507158 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b96a4-2602-4b9a-99b3-93380dc0dac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:00.507661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b96a4-2602-4b9a-99b3-93380dc0dac0 admin admin] PATH_INFO: `/v3/projects/40a538faf361474398f498b7347fc012/users/acfeb11b23b442dcb9f56349b1d1669d/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:00.529234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b96a4-2602-4b9a-99b3-93380dc0dac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a538faf361474398f498b7347fc012, user_id=acfeb11b23b442dcb9f56349b1d1669d, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:00.532035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b96a4-2602-4b9a-99b3-93380dc0dac0 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:00.549015 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1314/2657] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:57:00 2022] PUT /identity/v3/projects/40a538faf361474398f498b7347fc012/users/acfeb11b23b442dcb9f56349b1d1669d/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:00.553819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b634f7-c8fd-41be-85fe-b100a4dc907d admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:00.566016 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b634f7-c8fd-41be-85fe-b100a4dc907d admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:00.639361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863587e9-13bd-4251-9a83-18d7d9a9dcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:00.639957 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863587e9-13bd-4251-9a83-18d7d9a9dcc0 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:00.640240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863587e9-13bd-4251-9a83-18d7d9a9dcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:00.640522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863587e9-13bd-4251-9a83-18d7d9a9dcc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:00.641256 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863587e9-13bd-4251-9a83-18d7d9a9dcc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:00.642927 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863587e9-13bd-4251-9a83-18d7d9a9dcc0 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:00.654324 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1344/2658] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:57:00.658897 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b96a4-2602-4b9a-99b3-93380dc0dac0 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:00.671733 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b96a4-2602-4b9a-99b3-93380dc0dac0 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:00.677832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef00f2d-9c0e-400d-b382-4058df7f9e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:00.678750 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef00f2d-9c0e-400d-b382-4058df7f9e84 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:00.679034 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef00f2d-9c0e-400d-b382-4058df7f9e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:00.679308 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef00f2d-9c0e-400d-b382-4058df7f9e84 admin admin] PATH_INFO: `/v3/projects/40a538faf361474398f498b7347fc012/users/acfeb11b23b442dcb9f56349b1d1669d/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:00.681380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef00f2d-9c0e-400d-b382-4058df7f9e84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a538faf361474398f498b7347fc012, user_id=acfeb11b23b442dcb9f56349b1d1669d, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:00.684126 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef00f2d-9c0e-400d-b382-4058df7f9e84 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:00.696550 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1315/2659] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:57:00 2022] PUT /identity/v3/projects/40a538faf361474398f498b7347fc012/users/acfeb11b23b442dcb9f56349b1d1669d/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:00.708173 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda88ef-04af-49e7-afc8-6a8806410ac3 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:00.708471 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda88ef-04af-49e7-afc8-6a8806410ac3 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:00.708757 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda88ef-04af-49e7-afc8-6a8806410ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:00.739735 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-beda88ef-04af-49e7-afc8-6a8806410ac3 None None] MFA Rules not processed for user `acfeb11b23b442dcb9f56349b1d1669d`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:57:00.740972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-beda88ef-04af-49e7-afc8-6a8806410ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:00.841586 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1345/2660] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:57:00 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:57:00.860208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef00f2d-9c0e-400d-b382-4058df7f9e84 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:00.862621 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-fef00f2d-9c0e-400d-b382-4058df7f9e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:00.921765 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:00.962104 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef00f2d-9c0e-400d-b382-4058df7f9e84 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:00.966916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2471fd2-5068-4351-b783-af5fb3f5b7ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:00.968214 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2471fd2-5068-4351-b783-af5fb3f5b7ee service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:00.968680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2471fd2-5068-4351-b783-af5fb3f5b7ee service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:00.969102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2471fd2-5068-4351-b783-af5fb3f5b7ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:00.980843 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2471fd2-5068-4351-b783-af5fb3f5b7ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:00.982406 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2471fd2-5068-4351-b783-af5fb3f5b7ee service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:00.999531 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1316/2661] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:57:00 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:57:01.193052 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda88ef-04af-49e7-afc8-6a8806410ac3 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:01.204015 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda88ef-04af-49e7-afc8-6a8806410ac3 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:01.210192 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0732f2b-1b13-4224-9382-307844d608a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:01.210881 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0732f2b-1b13-4224-9382-307844d608a3 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:01.211169 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0732f2b-1b13-4224-9382-307844d608a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:01.211510 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0732f2b-1b13-4224-9382-307844d608a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:01.258372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:01.290765 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0732f2b-1b13-4224-9382-307844d608a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:01.292238 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0732f2b-1b13-4224-9382-307844d608a3 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:01.304097 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1346/2662] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:57:01 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:57:10.017708 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2471fd2-5068-4351-b783-af5fb3f5b7ee service cinder] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:10.105119 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2471fd2-5068-4351-b783-af5fb3f5b7ee service cinder] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:10.111059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac33fc4-d2bf-4651-9e20-5b55766a86fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:10.112003 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac33fc4-d2bf-4651-9e20-5b55766a86fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:10.112290 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac33fc4-d2bf-4651-9e20-5b55766a86fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:10.112575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac33fc4-d2bf-4651-9e20-5b55766a86fc admin admin] PATH_INFO: `/v3/users/7689026a1ac44b9aaa1dcfec0f79b0b4` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:10.114030 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac33fc4-d2bf-4651-9e20-5b55766a86fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7689026a1ac44b9aaa1dcfec0f79b0b4)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:10.116013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac33fc4-d2bf-4651-9e20-5b55766a86fc admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:10.186625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-cac33fc4-d2bf-4651-9e20-5b55766a86fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7689026a1ac44b9aaa1dcfec0f79b0b4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:10.212479 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-cac33fc4-d2bf-4651-9e20-5b55766a86fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7689026a1ac44b9aaa1dcfec0f79b0b4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:10.225844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-cac33fc4-d2bf-4651-9e20-5b55766a86fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7689026a1ac44b9aaa1dcfec0f79b0b4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:10.226712 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-cac33fc4-d2bf-4651-9e20-5b55766a86fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7689026a1ac44b9aaa1dcfec0f79b0b4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:10.227470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-cac33fc4-d2bf-4651-9e20-5b55766a86fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7689026a1ac44b9aaa1dcfec0f79b0b4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:10.243491 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1317/2663] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:57:10 2022] DELETE /identity/v3/users/7689026a1ac44b9aaa1dcfec0f79b0b4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:57:10.248266 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4fd0b-6838-443e-b673-6e007bfe044f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:10.248890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4fd0b-6838-443e-b673-6e007bfe044f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:10.248890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4fd0b-6838-443e-b673-6e007bfe044f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:10.269000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-c8d4fd0b-6838-443e-b673-6e007bfe044f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:57:10.271035 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-c8d4fd0b-6838-443e-b673-6e007bfe044f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:10.361600 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1347/2664] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:57:10.376114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac33fc4-d2bf-4651-9e20-5b55766a86fc admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:10.378527 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-cac33fc4-d2bf-4651-9e20-5b55766a86fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:10.497046 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac33fc4-d2bf-4651-9e20-5b55766a86fc admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:10.506903 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cce862-3ffb-403c-9621-ee09a158d5cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:10.508102 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cce862-3ffb-403c-9621-ee09a158d5cf service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:10.508660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cce862-3ffb-403c-9621-ee09a158d5cf service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:10.509171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cce862-3ffb-403c-9621-ee09a158d5cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:10.526838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cce862-3ffb-403c-9621-ee09a158d5cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:10.528958 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cce862-3ffb-403c-9621-ee09a158d5cf service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:10.537468 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:10.547223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1318/2665] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:57:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:57:11.016799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4fd0b-6838-443e-b673-6e007bfe044f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:11.019291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-c8d4fd0b-6838-443e-b673-6e007bfe044f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:11.047372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4fd0b-6838-443e-b673-6e007bfe044f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:11.056191 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9995e6-4c0b-4d4b-936a-06cd4061e7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:11.058000 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9995e6-4c0b-4d4b-936a-06cd4061e7e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:11.058499 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9995e6-4c0b-4d4b-936a-06cd4061e7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:11.059481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9995e6-4c0b-4d4b-936a-06cd4061e7e9 admin admin] PATH_INFO: `/v3/projects/2c41c1d9807e4d04a10148f55f2486c1` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:11.061291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9995e6-4c0b-4d4b-936a-06cd4061e7e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c41c1d9807e4d04a10148f55f2486c1)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:11.066094 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9995e6-4c0b-4d4b-936a-06cd4061e7e9 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:11.152616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ce9995e6-4c0b-4d4b-936a-06cd4061e7e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c41c1d9807e4d04a10148f55f2486c1', 'request_id': 'req-ce9995e6-4c0b-4d4b-936a-06cd4061e7e9'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:11.158851 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ce9995e6-4c0b-4d4b-936a-06cd4061e7e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c41c1d9807e4d04a10148f55f2486c1', 'request_id': 'req-ce9995e6-4c0b-4d4b-936a-06cd4061e7e9'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:11.169333 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ce9995e6-4c0b-4d4b-936a-06cd4061e7e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c41c1d9807e4d04a10148f55f2486c1', 'request_id': 'req-ce9995e6-4c0b-4d4b-936a-06cd4061e7e9'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:11.185358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ce9995e6-4c0b-4d4b-936a-06cd4061e7e9 admin admin] The token cache is being invalidate because project 2c41c1d9807e4d04a10148f55f2486c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:57:11.185935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ce9995e6-4c0b-4d4b-936a-06cd4061e7e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:11.187115 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-ce9995e6-4c0b-4d4b-936a-06cd4061e7e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:11.193212 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1348/2666] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:57:11 2022] DELETE /identity/v3/projects/2c41c1d9807e4d04a10148f55f2486c1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:11.198085 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cce862-3ffb-403c-9621-ee09a158d5cf service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:11.219045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cce862-3ffb-403c-9621-ee09a158d5cf service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:11.338947 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdd507-f286-441a-97c1-d86bd437a1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:11.339933 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdd507-f286-441a-97c1-d86bd437a1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:11.340214 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdd507-f286-441a-97c1-d86bd437a1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:11.340495 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdd507-f286-441a-97c1-d86bd437a1e4 admin admin] PATH_INFO: `/v3/users/acfeb11b23b442dcb9f56349b1d1669d` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:11.341895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bdd507-f286-441a-97c1-d86bd437a1e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acfeb11b23b442dcb9f56349b1d1669d)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:11.343909 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bdd507-f286-441a-97c1-d86bd437a1e4 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:11.421081 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f8bdd507-f286-441a-97c1-d86bd437a1e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acfeb11b23b442dcb9f56349b1d1669d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:11.445946 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f8bdd507-f286-441a-97c1-d86bd437a1e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acfeb11b23b442dcb9f56349b1d1669d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:11.456367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f8bdd507-f286-441a-97c1-d86bd437a1e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acfeb11b23b442dcb9f56349b1d1669d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:11.457138 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f8bdd507-f286-441a-97c1-d86bd437a1e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acfeb11b23b442dcb9f56349b1d1669d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:11.458113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f8bdd507-f286-441a-97c1-d86bd437a1e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acfeb11b23b442dcb9f56349b1d1669d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:11.475897 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1319/2667] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:57:11 2022] DELETE /identity/v3/users/acfeb11b23b442dcb9f56349b1d1669d => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:11.878258 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9995e6-4c0b-4d4b-936a-06cd4061e7e9 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:11.881322 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-ce9995e6-4c0b-4d4b-936a-06cd4061e7e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:11.898740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9995e6-4c0b-4d4b-936a-06cd4061e7e9 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:12.000866 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:12.017560 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb85de2-be51-4c70-b83c-879e0640e281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:12.019392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb85de2-be51-4c70-b83c-879e0640e281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:12.021263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb85de2-be51-4c70-b83c-879e0640e281 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:12.021984 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb85de2-be51-4c70-b83c-879e0640e281 admin admin] PATH_INFO: `/v3/projects/40a538faf361474398f498b7347fc012` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:12.025679 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb85de2-be51-4c70-b83c-879e0640e281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40a538faf361474398f498b7347fc012)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:12.029797 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb85de2-be51-4c70-b83c-879e0640e281 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:12.103526 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-2bb85de2-be51-4c70-b83c-879e0640e281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40a538faf361474398f498b7347fc012', 'request_id': 'req-2bb85de2-be51-4c70-b83c-879e0640e281'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:12.110949 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-2bb85de2-be51-4c70-b83c-879e0640e281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40a538faf361474398f498b7347fc012', 'request_id': 'req-2bb85de2-be51-4c70-b83c-879e0640e281'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:12.120082 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-2bb85de2-be51-4c70-b83c-879e0640e281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40a538faf361474398f498b7347fc012', 'request_id': 'req-2bb85de2-be51-4c70-b83c-879e0640e281'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:12.135096 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-2bb85de2-be51-4c70-b83c-879e0640e281 admin admin] The token cache is being invalidate because project 40a538faf361474398f498b7347fc012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:57:12.135439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-2bb85de2-be51-4c70-b83c-879e0640e281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:12.136181 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-2bb85de2-be51-4c70-b83c-879e0640e281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:12.138096 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1349/2668] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:57:11 2022] DELETE /identity/v3/projects/40a538faf361474398f498b7347fc012 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:12.143280 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffa3dd-11a5-415e-8ec2-67f698beea52 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:12.143573 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffa3dd-11a5-415e-8ec2-67f698beea52 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:12.143868 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffa3dd-11a5-415e-8ec2-67f698beea52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:12.171372 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-16ffa3dd-11a5-415e-8ec2-67f698beea52 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:57:12.173493 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-16ffa3dd-11a5-415e-8ec2-67f698beea52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:12.279137 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1320/2669] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:57:12.290398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242f2fe-7dd5-4af6-a8f8-53628fc07b8f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:12.290730 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242f2fe-7dd5-4af6-a8f8-53628fc07b8f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:12.291053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242f2fe-7dd5-4af6-a8f8-53628fc07b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:12.312420 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-c242f2fe-7dd5-4af6-a8f8-53628fc07b8f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:57:12.313920 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-c242f2fe-7dd5-4af6-a8f8-53628fc07b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:12.324294 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1350/2670] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:57:12.330653 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffa3dd-11a5-415e-8ec2-67f698beea52 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:12.342158 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffa3dd-11a5-415e-8ec2-67f698beea52 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:12.347403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad60e0-dde8-4aac-b641-811b69c0d036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:12.348934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad60e0-dde8-4aac-b641-811b69c0d036 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:12.349211 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad60e0-dde8-4aac-b641-811b69c0d036 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:12.349599 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad60e0-dde8-4aac-b641-811b69c0d036 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:12.350295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad60e0-dde8-4aac-b641-811b69c0d036 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:57:12.350607 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad60e0-dde8-4aac-b641-811b69c0d036 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:12.352303 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad60e0-dde8-4aac-b641-811b69c0d036 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:12.368262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1321/2671] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:57:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:57:12.373183 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242f2fe-7dd5-4af6-a8f8-53628fc07b8f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:12.384585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242f2fe-7dd5-4af6-a8f8-53628fc07b8f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:12.389818 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acc461-f105-476d-b6b2-1258da70ad3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:12.390528 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acc461-f105-476d-b6b2-1258da70ad3e admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:12.390808 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acc461-f105-476d-b6b2-1258da70ad3e admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:12.391081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acc461-f105-476d-b6b2-1258da70ad3e admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:12.391799 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01acc461-f105-476d-b6b2-1258da70ad3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:12.393844 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01acc461-f105-476d-b6b2-1258da70ad3e admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:12.425709 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1351/2672] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:57:12 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:57:12.431450 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad60e0-dde8-4aac-b641-811b69c0d036 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:12.438872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:12.444197 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad60e0-dde8-4aac-b641-811b69c0d036 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:12.519648 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718291e-f5df-47b8-980d-18c2d32b0da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:12.521064 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718291e-f5df-47b8-980d-18c2d32b0da5 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:12.521496 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718291e-f5df-47b8-980d-18c2d32b0da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:12.521758 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718291e-f5df-47b8-980d-18c2d32b0da5 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:12.522376 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718291e-f5df-47b8-980d-18c2d32b0da5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:12.524390 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718291e-f5df-47b8-980d-18c2d32b0da5 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:12.554948 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1322/2673] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:57:12 2022] POST /identity/v3/users => generated 291 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:57:12.559228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acc461-f105-476d-b6b2-1258da70ad3e admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:12.571240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acc461-f105-476d-b6b2-1258da70ad3e admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:12.576279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd5805-3e6a-4e9f-91b2-5c7d7df77251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:12.576973 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd5805-3e6a-4e9f-91b2-5c7d7df77251 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:12.577261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd5805-3e6a-4e9f-91b2-5c7d7df77251 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:12.577540 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd5805-3e6a-4e9f-91b2-5c7d7df77251 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:12.578330 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd5805-3e6a-4e9f-91b2-5c7d7df77251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:12.580003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd5805-3e6a-4e9f-91b2-5c7d7df77251 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:12.593500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1352/2674] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:57:12.599723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718291e-f5df-47b8-980d-18c2d32b0da5 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:12.612680 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718291e-f5df-47b8-980d-18c2d32b0da5 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:12.622336 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba4277-78f2-4dba-a91f-75c06756da20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:12.623513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba4277-78f2-4dba-a91f-75c06756da20 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:12.624016 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba4277-78f2-4dba-a91f-75c06756da20 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:12.624509 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba4277-78f2-4dba-a91f-75c06756da20 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:12.625963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba4277-78f2-4dba-a91f-75c06756da20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:12.629566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba4277-78f2-4dba-a91f-75c06756da20 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:12.650539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1323/2675] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:57:12.655435 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd5805-3e6a-4e9f-91b2-5c7d7df77251 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:12.669781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd5805-3e6a-4e9f-91b2-5c7d7df77251 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:12.674972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7bad0c-f7f5-445c-8164-d3b762a5b3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:12.676251 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7bad0c-f7f5-445c-8164-d3b762a5b3a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:12.676549 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7bad0c-f7f5-445c-8164-d3b762a5b3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:12.676834 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7bad0c-f7f5-445c-8164-d3b762a5b3a1 admin admin] PATH_INFO: `/v3/projects/6c28c036e381477aa9a0cf91c4838834/users/8d7e925e40a04eb7b1a62644f18e479f/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:12.694611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7bad0c-f7f5-445c-8164-d3b762a5b3a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c28c036e381477aa9a0cf91c4838834, user_id=8d7e925e40a04eb7b1a62644f18e479f, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:12.697726 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7bad0c-f7f5-445c-8164-d3b762a5b3a1 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:12.717967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1353/2676] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:57:12 2022] PUT /identity/v3/projects/6c28c036e381477aa9a0cf91c4838834/users/8d7e925e40a04eb7b1a62644f18e479f/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:57:12.731432 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53871dde-84e8-4059-a67b-ad1f704bb2ca None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:12.731770 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53871dde-84e8-4059-a67b-ad1f704bb2ca None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:12.732127 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53871dde-84e8-4059-a67b-ad1f704bb2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:12.764832 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-53871dde-84e8-4059-a67b-ad1f704bb2ca None None] MFA Rules not processed for user `8d7e925e40a04eb7b1a62644f18e479f`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:57:12.766442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-53871dde-84e8-4059-a67b-ad1f704bb2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:12.880395 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1324/2677] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:57:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:57:12.885219 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7bad0c-f7f5-445c-8164-d3b762a5b3a1 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:12.960702 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7bad0c-f7f5-445c-8164-d3b762a5b3a1 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:12.966273 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2b3f7-d208-41d5-ad59-b6fd28095774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:12.967187 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2b3f7-d208-41d5-ad59-b6fd28095774 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:12.967675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2b3f7-d208-41d5-ad59-b6fd28095774 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:12.968365 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2b3f7-d208-41d5-ad59-b6fd28095774 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:12.969533 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2b3f7-d208-41d5-ad59-b6fd28095774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:12.974573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2b3f7-d208-41d5-ad59-b6fd28095774 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:13.021375 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1354/2678] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:57:12 2022] POST /identity/v3/projects => generated 377 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:57:13.025906 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53871dde-84e8-4059-a67b-ad1f704bb2ca None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:13.039448 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53871dde-84e8-4059-a67b-ad1f704bb2ca None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:13.112323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fad624-4628-422f-a129-f1e8ca6fa73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:13.113043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fad624-4628-422f-a129-f1e8ca6fa73e admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:13.113337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fad624-4628-422f-a129-f1e8ca6fa73e admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:13.113993 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fad624-4628-422f-a129-f1e8ca6fa73e admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:13.114640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fad624-4628-422f-a129-f1e8ca6fa73e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:13.116690 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fad624-4628-422f-a129-f1e8ca6fa73e admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:13.142541 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1325/2679] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:57:13 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:57:13.149189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2b3f7-d208-41d5-ad59-b6fd28095774 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:13.161895 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2b3f7-d208-41d5-ad59-b6fd28095774 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:13.166954 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163c8ee-9f3a-4d68-b043-3288a6a1577a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:13.167648 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163c8ee-9f3a-4d68-b043-3288a6a1577a admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:13.167935 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163c8ee-9f3a-4d68-b043-3288a6a1577a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:13.168213 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163c8ee-9f3a-4d68-b043-3288a6a1577a admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:13.168962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163c8ee-9f3a-4d68-b043-3288a6a1577a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:13.170931 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163c8ee-9f3a-4d68-b043-3288a6a1577a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:13.172566 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:13.186737 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1355/2680] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:57:13.193525 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fad624-4628-422f-a129-f1e8ca6fa73e admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:13.199323 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:13.208305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fad624-4628-422f-a129-f1e8ca6fa73e admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:13.214039 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e77849-0ab8-4154-a78c-c43225717c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:13.214969 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e77849-0ab8-4154-a78c-c43225717c39 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:13.215298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e77849-0ab8-4154-a78c-c43225717c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:13.215571 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e77849-0ab8-4154-a78c-c43225717c39 admin admin] PATH_INFO: `/v3/projects/536b8f9251f14fdfb424402058bf8bd9/users/06539c77994048e9a867b17a59ae4f40/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:13.234979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e77849-0ab8-4154-a78c-c43225717c39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=536b8f9251f14fdfb424402058bf8bd9, user_id=06539c77994048e9a867b17a59ae4f40, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:13.237795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e77849-0ab8-4154-a78c-c43225717c39 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:13.255581 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1326/2681] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:57:13 2022] PUT /identity/v3/projects/536b8f9251f14fdfb424402058bf8bd9/users/06539c77994048e9a867b17a59ae4f40/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:13.259888 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163c8ee-9f3a-4d68-b043-3288a6a1577a admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:13.271056 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163c8ee-9f3a-4d68-b043-3288a6a1577a admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:13.342544 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5c640-1f5e-4a75-967e-c721a409a76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:13.342994 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5c640-1f5e-4a75-967e-c721a409a76f admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:13.343286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5c640-1f5e-4a75-967e-c721a409a76f admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:13.343606 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5c640-1f5e-4a75-967e-c721a409a76f admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:13.344506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5c640-1f5e-4a75-967e-c721a409a76f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:13.346404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5c640-1f5e-4a75-967e-c721a409a76f admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:13.365745 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1356/2682] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:57:13.370353 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e77849-0ab8-4154-a78c-c43225717c39 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:13.382251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e77849-0ab8-4154-a78c-c43225717c39 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:13.387284 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a46ade-bd10-4064-acca-0d16c3db818d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:13.388376 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a46ade-bd10-4064-acca-0d16c3db818d admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:13.388663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a46ade-bd10-4064-acca-0d16c3db818d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:13.388941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a46ade-bd10-4064-acca-0d16c3db818d admin admin] PATH_INFO: `/v3/projects/536b8f9251f14fdfb424402058bf8bd9/users/06539c77994048e9a867b17a59ae4f40/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:13.391144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a46ade-bd10-4064-acca-0d16c3db818d admin admin] RBAC: Authorizing `identity:create_grant(project_id=536b8f9251f14fdfb424402058bf8bd9, user_id=06539c77994048e9a867b17a59ae4f40, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:13.393915 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a46ade-bd10-4064-acca-0d16c3db818d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:13.406882 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1327/2683] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:57:13 2022] PUT /identity/v3/projects/536b8f9251f14fdfb424402058bf8bd9/users/06539c77994048e9a867b17a59ae4f40/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:57:13.419637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f156f-deee-40dd-ac65-e02cc9f2c8d1 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:13.420190 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f156f-deee-40dd-ac65-e02cc9f2c8d1 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:13.420473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f156f-deee-40dd-ac65-e02cc9f2c8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:13.453286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-0a7f156f-deee-40dd-ac65-e02cc9f2c8d1 None None] MFA Rules not processed for user `06539c77994048e9a867b17a59ae4f40`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:57:13.454605 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0a7f156f-deee-40dd-ac65-e02cc9f2c8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:13.556309 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1357/2684] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:57:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:57:13.571645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a46ade-bd10-4064-acca-0d16c3db818d admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:13.573608 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d2a46ade-bd10-4064-acca-0d16c3db818d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:13.649183 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a46ade-bd10-4064-acca-0d16c3db818d admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:13.653033 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de377be4-3824-47de-80bc-f755497ea8c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:13.653763 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de377be4-3824-47de-80bc-f755497ea8c8 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:13.654053 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de377be4-3824-47de-80bc-f755497ea8c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:13.654331 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de377be4-3824-47de-80bc-f755497ea8c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:13.665838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de377be4-3824-47de-80bc-f755497ea8c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:13.667354 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de377be4-3824-47de-80bc-f755497ea8c8 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:13.679695 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1328/2685] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:57:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:57:14.114453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f156f-deee-40dd-ac65-e02cc9f2c8d1 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:14.197140 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f156f-deee-40dd-ac65-e02cc9f2c8d1 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:14.202290 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecdd0f7-799d-4ddc-a79a-74c01518ecf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:14.203189 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdd0f7-799d-4ddc-a79a-74c01518ecf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:14.203470 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdd0f7-799d-4ddc-a79a-74c01518ecf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:14.203743 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdd0f7-799d-4ddc-a79a-74c01518ecf8 admin admin] PATH_INFO: `/v3/users/8d7e925e40a04eb7b1a62644f18e479f` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:14.205057 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecdd0f7-799d-4ddc-a79a-74c01518ecf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d7e925e40a04eb7b1a62644f18e479f)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:14.207598 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecdd0f7-799d-4ddc-a79a-74c01518ecf8 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:14.232857 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:14.264210 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-7ecdd0f7-799d-4ddc-a79a-74c01518ecf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d7e925e40a04eb7b1a62644f18e479f'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:14.279841 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-7ecdd0f7-799d-4ddc-a79a-74c01518ecf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d7e925e40a04eb7b1a62644f18e479f'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:14.280611 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-7ecdd0f7-799d-4ddc-a79a-74c01518ecf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d7e925e40a04eb7b1a62644f18e479f'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:14.291427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-7ecdd0f7-799d-4ddc-a79a-74c01518ecf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d7e925e40a04eb7b1a62644f18e479f'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:14.301119 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-7ecdd0f7-799d-4ddc-a79a-74c01518ecf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d7e925e40a04eb7b1a62644f18e479f'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:14.308355 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1358/2686] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:57:14 2022] DELETE /identity/v3/users/8d7e925e40a04eb7b1a62644f18e479f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:57:14.313296 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a293a0-0a81-4d27-b255-da66afc43afb None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:14.313929 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a293a0-0a81-4d27-b255-da66afc43afb None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:14.314389 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a293a0-0a81-4d27-b255-da66afc43afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:14.336957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-23a293a0-0a81-4d27-b255-da66afc43afb None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:57:14.338471 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-23a293a0-0a81-4d27-b255-da66afc43afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:14.414538 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1329/2687] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:57:14.425593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecdd0f7-799d-4ddc-a79a-74c01518ecf8 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:14.427838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-7ecdd0f7-799d-4ddc-a79a-74c01518ecf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:14.499408 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecdd0f7-799d-4ddc-a79a-74c01518ecf8 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:14.503650 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35da6a3-6738-4abd-b6da-8e955f46cb63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:14.505138 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35da6a3-6738-4abd-b6da-8e955f46cb63 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:14.505487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35da6a3-6738-4abd-b6da-8e955f46cb63 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:14.505817 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35da6a3-6738-4abd-b6da-8e955f46cb63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:14.517201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35da6a3-6738-4abd-b6da-8e955f46cb63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:14.518691 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35da6a3-6738-4abd-b6da-8e955f46cb63 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:14.531232 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1359/2688] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:57:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:57:14.854257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a293a0-0a81-4d27-b255-da66afc43afb None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:14.855875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-23a293a0-0a81-4d27-b255-da66afc43afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:14.872479 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a293a0-0a81-4d27-b255-da66afc43afb None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:14.877612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e4b4c-4802-400e-aba2-8b63dce88f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:14.878541 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e4b4c-4802-400e-aba2-8b63dce88f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:14.878826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e4b4c-4802-400e-aba2-8b63dce88f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:14.879687 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e4b4c-4802-400e-aba2-8b63dce88f4d admin admin] PATH_INFO: `/v3/projects/6c28c036e381477aa9a0cf91c4838834` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:14.880954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e4b4c-4802-400e-aba2-8b63dce88f4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c28c036e381477aa9a0cf91c4838834)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:14.883006 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e4b4c-4802-400e-aba2-8b63dce88f4d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:14.944838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-521e4b4c-4802-400e-aba2-8b63dce88f4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c28c036e381477aa9a0cf91c4838834', 'request_id': 'req-521e4b4c-4802-400e-aba2-8b63dce88f4d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:14.952519 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-521e4b4c-4802-400e-aba2-8b63dce88f4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c28c036e381477aa9a0cf91c4838834', 'request_id': 'req-521e4b4c-4802-400e-aba2-8b63dce88f4d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:14.961035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-521e4b4c-4802-400e-aba2-8b63dce88f4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c28c036e381477aa9a0cf91c4838834', 'request_id': 'req-521e4b4c-4802-400e-aba2-8b63dce88f4d'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:14.963484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:14.989114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-521e4b4c-4802-400e-aba2-8b63dce88f4d admin admin] The token cache is being invalidate because project 6c28c036e381477aa9a0cf91c4838834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:57:14.989555 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-521e4b4c-4802-400e-aba2-8b63dce88f4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:14.990640 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-521e4b4c-4802-400e-aba2-8b63dce88f4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:14.992550 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1330/2689] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:57:14 2022] DELETE /identity/v3/projects/6c28c036e381477aa9a0cf91c4838834 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:14.996971 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35da6a3-6738-4abd-b6da-8e955f46cb63 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:15.008974 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35da6a3-6738-4abd-b6da-8e955f46cb63 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:15.079004 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e7f0f-e27b-4575-bdcd-096dca5817d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:15.079912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e7f0f-e27b-4575-bdcd-096dca5817d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:15.080199 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e7f0f-e27b-4575-bdcd-096dca5817d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:15.080478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e7f0f-e27b-4575-bdcd-096dca5817d2 admin admin] PATH_INFO: `/v3/users/06539c77994048e9a867b17a59ae4f40` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:15.081785 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e7f0f-e27b-4575-bdcd-096dca5817d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06539c77994048e9a867b17a59ae4f40)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:15.083768 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e7f0f-e27b-4575-bdcd-096dca5817d2 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:15.138377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:15.146977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-a18e7f0f-e27b-4575-bdcd-096dca5817d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06539c77994048e9a867b17a59ae4f40'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:15.162983 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-a18e7f0f-e27b-4575-bdcd-096dca5817d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06539c77994048e9a867b17a59ae4f40'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:15.163728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-a18e7f0f-e27b-4575-bdcd-096dca5817d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06539c77994048e9a867b17a59ae4f40'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:15.173677 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-a18e7f0f-e27b-4575-bdcd-096dca5817d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06539c77994048e9a867b17a59ae4f40'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:15.183701 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-a18e7f0f-e27b-4575-bdcd-096dca5817d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06539c77994048e9a867b17a59ae4f40'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:15.190890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1360/2690] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:57:14 2022] DELETE /identity/v3/users/06539c77994048e9a867b17a59ae4f40 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:57:15.517010 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e4b4c-4802-400e-aba2-8b63dce88f4d admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:15.519378 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-521e4b4c-4802-400e-aba2-8b63dce88f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:15.540351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e4b4c-4802-400e-aba2-8b63dce88f4d admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:15.623551 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa816dc0-ee63-4e40-b930-f8199d8bdbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:15.624363 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa816dc0-ee63-4e40-b930-f8199d8bdbf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:15.624646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa816dc0-ee63-4e40-b930-f8199d8bdbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:15.624957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa816dc0-ee63-4e40-b930-f8199d8bdbf2 admin admin] PATH_INFO: `/v3/projects/536b8f9251f14fdfb424402058bf8bd9` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:15.626327 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa816dc0-ee63-4e40-b930-f8199d8bdbf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=536b8f9251f14fdfb424402058bf8bd9)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:15.628405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa816dc0-ee63-4e40-b930-f8199d8bdbf2 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:15.685177 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-aa816dc0-ee63-4e40-b930-f8199d8bdbf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '536b8f9251f14fdfb424402058bf8bd9', 'request_id': 'req-aa816dc0-ee63-4e40-b930-f8199d8bdbf2'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:15.693569 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-aa816dc0-ee63-4e40-b930-f8199d8bdbf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '536b8f9251f14fdfb424402058bf8bd9', 'request_id': 'req-aa816dc0-ee63-4e40-b930-f8199d8bdbf2'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:15.700297 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-aa816dc0-ee63-4e40-b930-f8199d8bdbf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '536b8f9251f14fdfb424402058bf8bd9', 'request_id': 'req-aa816dc0-ee63-4e40-b930-f8199d8bdbf2'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:15.715249 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-aa816dc0-ee63-4e40-b930-f8199d8bdbf2 admin admin] The token cache is being invalidate because project 536b8f9251f14fdfb424402058bf8bd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:57:15.715542 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-aa816dc0-ee63-4e40-b930-f8199d8bdbf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:15.716268 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-aa816dc0-ee63-4e40-b930-f8199d8bdbf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:15.718024 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1331/2691] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:57:15 2022] DELETE /identity/v3/projects/536b8f9251f14fdfb424402058bf8bd9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:15.725095 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433df4d-55cb-4c10-92a4-534f147881b3 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:15.725433 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433df4d-55cb-4c10-92a4-534f147881b3 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:15.725806 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433df4d-55cb-4c10-92a4-534f147881b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:15.749319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-d433df4d-55cb-4c10-92a4-534f147881b3 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:57:15.751490 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-d433df4d-55cb-4c10-92a4-534f147881b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:15.865843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1361/2692] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:57:15.875881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015f023-6c61-47c5-b149-8b9f3667b8eb None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:15.876131 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015f023-6c61-47c5-b149-8b9f3667b8eb None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:15.876446 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015f023-6c61-47c5-b149-8b9f3667b8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:15.898660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-c015f023-6c61-47c5-b149-8b9f3667b8eb None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:57:15.900010 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-c015f023-6c61-47c5-b149-8b9f3667b8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:15.909657 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1332/2693] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:57:15.914940 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433df4d-55cb-4c10-92a4-534f147881b3 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:15.926817 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433df4d-55cb-4c10-92a4-534f147881b3 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:15.932526 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df7f3d-259c-4394-b942-cf40244733d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:15.933300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df7f3d-259c-4394-b942-cf40244733d8 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:15.933737 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df7f3d-259c-4394-b942-cf40244733d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:15.934105 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df7f3d-259c-4394-b942-cf40244733d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:15.934979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df7f3d-259c-4394-b942-cf40244733d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:57:15.935398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df7f3d-259c-4394-b942-cf40244733d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:15.937171 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df7f3d-259c-4394-b942-cf40244733d8 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:15.951376 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1362/2694] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:57:15.956049 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015f023-6c61-47c5-b149-8b9f3667b8eb None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:15.966823 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015f023-6c61-47c5-b149-8b9f3667b8eb None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:15.972255 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180ae6a-0013-4c84-9cea-29ba6689963a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:15.973688 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180ae6a-0013-4c84-9cea-29ba6689963a admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:15.973980 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180ae6a-0013-4c84-9cea-29ba6689963a admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:15.974263 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180ae6a-0013-4c84-9cea-29ba6689963a admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:15.975078 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7180ae6a-0013-4c84-9cea-29ba6689963a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:15.977158 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7180ae6a-0013-4c84-9cea-29ba6689963a admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:16.009227 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1333/2695] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:57:15 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:57:16.013994 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df7f3d-259c-4394-b942-cf40244733d8 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:16.026885 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df7f3d-259c-4394-b942-cf40244733d8 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:16.103139 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae1aac-9c98-4e54-a928-4fac8c351b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:16.103828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae1aac-9c98-4e54-a928-4fac8c351b1e admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:16.104117 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae1aac-9c98-4e54-a928-4fac8c351b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:16.104395 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae1aac-9c98-4e54-a928-4fac8c351b1e admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:16.105117 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae1aac-9c98-4e54-a928-4fac8c351b1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:16.107110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae1aac-9c98-4e54-a928-4fac8c351b1e admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:16.130321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1363/2696] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:57:16 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:57:16.136502 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180ae6a-0013-4c84-9cea-29ba6689963a admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:16.149017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180ae6a-0013-4c84-9cea-29ba6689963a admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:16.154195 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb17f1-7ed0-4b21-893b-76f0b321ccda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:16.154888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb17f1-7ed0-4b21-893b-76f0b321ccda admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:16.155176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb17f1-7ed0-4b21-893b-76f0b321ccda admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:16.155472 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb17f1-7ed0-4b21-893b-76f0b321ccda admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:16.156201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb17f1-7ed0-4b21-893b-76f0b321ccda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:16.157885 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb17f1-7ed0-4b21-893b-76f0b321ccda admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:16.172159 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1334/2697] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:57:16.178092 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae1aac-9c98-4e54-a928-4fac8c351b1e admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:16.197024 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae1aac-9c98-4e54-a928-4fac8c351b1e admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:16.205538 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f0778-f5de-46d9-aa45-4d4500eef0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:16.206735 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f0778-f5de-46d9-aa45-4d4500eef0bd admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:16.207907 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f0778-f5de-46d9-aa45-4d4500eef0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:16.208599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f0778-f5de-46d9-aa45-4d4500eef0bd admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:16.209697 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f0778-f5de-46d9-aa45-4d4500eef0bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:16.212838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265f0778-f5de-46d9-aa45-4d4500eef0bd admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:16.226828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1364/2698] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:57:16.232901 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb17f1-7ed0-4b21-893b-76f0b321ccda admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:16.244560 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb17f1-7ed0-4b21-893b-76f0b321ccda admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:16.249828 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5456e28-5f31-4316-927b-fb2d78c7379b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:16.250761 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5456e28-5f31-4316-927b-fb2d78c7379b admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:16.251054 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5456e28-5f31-4316-927b-fb2d78c7379b admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:16.251333 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5456e28-5f31-4316-927b-fb2d78c7379b admin admin] PATH_INFO: `/v3/projects/3a18e40060604986a82e5e356ab42ac1/users/eb677fc6ed5f47d1a4c3ab3e8b09f3ad/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:16.271419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5456e28-5f31-4316-927b-fb2d78c7379b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a18e40060604986a82e5e356ab42ac1, user_id=eb677fc6ed5f47d1a4c3ab3e8b09f3ad, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:16.274330 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5456e28-5f31-4316-927b-fb2d78c7379b admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:16.292687 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1335/2699] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:57:16 2022] PUT /identity/v3/projects/3a18e40060604986a82e5e356ab42ac1/users/eb677fc6ed5f47d1a4c3ab3e8b09f3ad/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:16.304412 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b570dc-f38d-44a7-a2bd-54ff851a1c13 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:16.304798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b570dc-f38d-44a7-a2bd-54ff851a1c13 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:16.305184 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b570dc-f38d-44a7-a2bd-54ff851a1c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:16.339369 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-b9b570dc-f38d-44a7-a2bd-54ff851a1c13 None None] MFA Rules not processed for user `eb677fc6ed5f47d1a4c3ab3e8b09f3ad`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:57:16.340814 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-b9b570dc-f38d-44a7-a2bd-54ff851a1c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:16.358104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:16.461458 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1365/2700] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:57:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:57:16.466091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5456e28-5f31-4316-927b-fb2d78c7379b admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:16.490757 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:16.548349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5456e28-5f31-4316-927b-fb2d78c7379b admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:16.555345 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfde608-5d06-4472-8d02-d78979590d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:16.556103 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfde608-5d06-4472-8d02-d78979590d34 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:16.556435 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfde608-5d06-4472-8d02-d78979590d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:16.556762 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfde608-5d06-4472-8d02-d78979590d34 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:16.557558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfde608-5d06-4472-8d02-d78979590d34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:16.559629 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfde608-5d06-4472-8d02-d78979590d34 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:16.591232 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1336/2701] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:57:16 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:57:16.597339 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b570dc-f38d-44a7-a2bd-54ff851a1c13 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:16.610592 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b570dc-f38d-44a7-a2bd-54ff851a1c13 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:16.700128 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35e49e-61aa-4e4f-b8f6-3e1084a02516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:16.701309 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35e49e-61aa-4e4f-b8f6-3e1084a02516 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:16.701834 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35e49e-61aa-4e4f-b8f6-3e1084a02516 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:16.702320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35e49e-61aa-4e4f-b8f6-3e1084a02516 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:16.703608 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35e49e-61aa-4e4f-b8f6-3e1084a02516 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:16.707897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35e49e-61aa-4e4f-b8f6-3e1084a02516 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:16.742588 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1366/2702] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:57:16 2022] POST /identity/v3/users => generated 298 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:57:16.747254 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfde608-5d06-4472-8d02-d78979590d34 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:16.758710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfde608-5d06-4472-8d02-d78979590d34 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:16.763766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1da5e-f07d-4342-bfb3-340d982f91de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:16.764455 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1da5e-f07d-4342-bfb3-340d982f91de admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:16.764780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1da5e-f07d-4342-bfb3-340d982f91de admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:16.765059 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1da5e-f07d-4342-bfb3-340d982f91de admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:16.765847 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1da5e-f07d-4342-bfb3-340d982f91de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:16.767508 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1da5e-f07d-4342-bfb3-340d982f91de admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:16.786208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1337/2703] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:57:16.791194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35e49e-61aa-4e4f-b8f6-3e1084a02516 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:16.803763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35e49e-61aa-4e4f-b8f6-3e1084a02516 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:16.809378 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2609810-e175-456d-bcb6-cb806bdd4343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:16.810346 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2609810-e175-456d-bcb6-cb806bdd4343 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:16.810632 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2609810-e175-456d-bcb6-cb806bdd4343 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:16.810907 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2609810-e175-456d-bcb6-cb806bdd4343 admin admin] PATH_INFO: `/v3/projects/f51c227d542d4b50ba7a2202f37acead/users/ea205849757643dd9628551a3877050e/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:16.833489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2609810-e175-456d-bcb6-cb806bdd4343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f51c227d542d4b50ba7a2202f37acead, user_id=ea205849757643dd9628551a3877050e, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:16.836682 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2609810-e175-456d-bcb6-cb806bdd4343 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:16.854191 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1367/2704] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:57:16 2022] PUT /identity/v3/projects/f51c227d542d4b50ba7a2202f37acead/users/ea205849757643dd9628551a3877050e/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:57:16.858690 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1da5e-f07d-4342-bfb3-340d982f91de admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:16.871383 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1da5e-f07d-4342-bfb3-340d982f91de admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:16.946100 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f35eb-6e39-4502-b4c0-ebfa51962510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:16.946799 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f35eb-6e39-4502-b4c0-ebfa51962510 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:16.947086 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f35eb-6e39-4502-b4c0-ebfa51962510 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:16.947365 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f35eb-6e39-4502-b4c0-ebfa51962510 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:16.948114 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f35eb-6e39-4502-b4c0-ebfa51962510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:16.949826 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f35eb-6e39-4502-b4c0-ebfa51962510 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:16.961893 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1338/2705] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:57:16.968014 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2609810-e175-456d-bcb6-cb806bdd4343 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:16.981204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2609810-e175-456d-bcb6-cb806bdd4343 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:16.986636 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad271e-2b84-4662-aa6e-b1c72eb422f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:16.987568 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad271e-2b84-4662-aa6e-b1c72eb422f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:16.987858 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad271e-2b84-4662-aa6e-b1c72eb422f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:16.988137 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad271e-2b84-4662-aa6e-b1c72eb422f2 admin admin] PATH_INFO: `/v3/projects/f51c227d542d4b50ba7a2202f37acead/users/ea205849757643dd9628551a3877050e/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:16.991823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad271e-2b84-4662-aa6e-b1c72eb422f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f51c227d542d4b50ba7a2202f37acead, user_id=ea205849757643dd9628551a3877050e, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:16.994763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad271e-2b84-4662-aa6e-b1c72eb422f2 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:17.010947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1368/2706] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:57:16 2022] PUT /identity/v3/projects/f51c227d542d4b50ba7a2202f37acead/users/ea205849757643dd9628551a3877050e/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:17.031449 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed397f6-7c8a-429a-bd5e-755a21b74c78 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:17.031772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed397f6-7c8a-429a-bd5e-755a21b74c78 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:17.032055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed397f6-7c8a-429a-bd5e-755a21b74c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:17.066194 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-bed397f6-7c8a-429a-bd5e-755a21b74c78 None None] MFA Rules not processed for user `ea205849757643dd9628551a3877050e`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:57:17.068276 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-bed397f6-7c8a-429a-bd5e-755a21b74c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:17.189209 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1339/2707] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:57:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:57:17.206894 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad271e-2b84-4662-aa6e-b1c72eb422f2 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:17.209470 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-b5ad271e-2b84-4662-aa6e-b1c72eb422f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:17.284078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad271e-2b84-4662-aa6e-b1c72eb422f2 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:17.289422 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a2b7d-6099-465a-8119-7d9b50710526 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:17.290146 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a2b7d-6099-465a-8119-7d9b50710526 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:17.290433 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a2b7d-6099-465a-8119-7d9b50710526 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:17.290717 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a2b7d-6099-465a-8119-7d9b50710526 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:17.306041 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a2b7d-6099-465a-8119-7d9b50710526 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:17.307855 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a2b7d-6099-465a-8119-7d9b50710526 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:17.321793 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1369/2708] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:57:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:57:17.427263 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed397f6-7c8a-429a-bd5e-755a21b74c78 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:17.445312 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed397f6-7c8a-429a-bd5e-755a21b74c78 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:17.452149 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6cbaa-2c54-4eb2-94d1-43c39d4b6c5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:17.453321 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6cbaa-2c54-4eb2-94d1-43c39d4b6c5a service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:17.453891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6cbaa-2c54-4eb2-94d1-43c39d4b6c5a service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:17.454408 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6cbaa-2c54-4eb2-94d1-43c39d4b6c5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:17.478892 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:17.572646 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6cbaa-2c54-4eb2-94d1-43c39d4b6c5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:17.575325 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6cbaa-2c54-4eb2-94d1-43c39d4b6c5a service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:17.596304 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1340/2709] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:57:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:57:18.010049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a2b7d-6099-465a-8119-7d9b50710526 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:18.025736 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:18.101284 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a2b7d-6099-465a-8119-7d9b50710526 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:18.106712 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ce647-054a-4406-aed8-86f56cb83c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:18.107638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ce647-054a-4406-aed8-86f56cb83c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:18.107928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ce647-054a-4406-aed8-86f56cb83c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:18.108216 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ce647-054a-4406-aed8-86f56cb83c14 admin admin] PATH_INFO: `/v3/users/eb677fc6ed5f47d1a4c3ab3e8b09f3ad` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:18.109724 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ce647-054a-4406-aed8-86f56cb83c14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb677fc6ed5f47d1a4c3ab3e8b09f3ad)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:18.111763 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ce647-054a-4406-aed8-86f56cb83c14 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:18.191933 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-030ce647-054a-4406-aed8-86f56cb83c14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb677fc6ed5f47d1a4c3ab3e8b09f3ad'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:18.225946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-030ce647-054a-4406-aed8-86f56cb83c14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb677fc6ed5f47d1a4c3ab3e8b09f3ad'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:18.227404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-030ce647-054a-4406-aed8-86f56cb83c14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb677fc6ed5f47d1a4c3ab3e8b09f3ad'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:18.240781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-030ce647-054a-4406-aed8-86f56cb83c14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb677fc6ed5f47d1a4c3ab3e8b09f3ad'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:18.255699 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-030ce647-054a-4406-aed8-86f56cb83c14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb677fc6ed5f47d1a4c3ab3e8b09f3ad'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:18.268754 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1370/2710] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:57:18 2022] DELETE /identity/v3/users/eb677fc6ed5f47d1a4c3ab3e8b09f3ad => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:18.276454 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de53bfc-866b-48f2-8fcc-5629a064fa7f None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:18.276950 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de53bfc-866b-48f2-8fcc-5629a064fa7f None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:18.277433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de53bfc-866b-48f2-8fcc-5629a064fa7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:18.303115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-3de53bfc-866b-48f2-8fcc-5629a064fa7f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:57:18.304357 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3de53bfc-866b-48f2-8fcc-5629a064fa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:18.395785 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1341/2711] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:57:18.407016 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ce647-054a-4406-aed8-86f56cb83c14 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:18.408810 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-030ce647-054a-4406-aed8-86f56cb83c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:18.485983 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ce647-054a-4406-aed8-86f56cb83c14 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:18.490917 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad6207-ae4e-47b4-b7f0-6c4258abfce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:18.491603 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad6207-ae4e-47b4-b7f0-6c4258abfce1 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:18.491887 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad6207-ae4e-47b4-b7f0-6c4258abfce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:18.492167 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad6207-ae4e-47b4-b7f0-6c4258abfce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:18.502435 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad6207-ae4e-47b4-b7f0-6c4258abfce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:18.504475 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad6207-ae4e-47b4-b7f0-6c4258abfce1 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:18.515986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1371/2712] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:57:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:57:18.899559 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de53bfc-866b-48f2-8fcc-5629a064fa7f None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:18.901689 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3de53bfc-866b-48f2-8fcc-5629a064fa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:18.915480 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de53bfc-866b-48f2-8fcc-5629a064fa7f None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:18.921103 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e86861-ea77-4fa6-b9e0-d3448ae54df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:18.922049 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e86861-ea77-4fa6-b9e0-d3448ae54df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:18.922336 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e86861-ea77-4fa6-b9e0-d3448ae54df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:18.922618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e86861-ea77-4fa6-b9e0-d3448ae54df4 admin admin] PATH_INFO: `/v3/projects/3a18e40060604986a82e5e356ab42ac1` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:18.924044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e86861-ea77-4fa6-b9e0-d3448ae54df4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a18e40060604986a82e5e356ab42ac1)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:18.926625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e86861-ea77-4fa6-b9e0-d3448ae54df4 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:18.988686 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-47e86861-ea77-4fa6-b9e0-d3448ae54df4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a18e40060604986a82e5e356ab42ac1', 'request_id': 'req-47e86861-ea77-4fa6-b9e0-d3448ae54df4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:18.996078 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-47e86861-ea77-4fa6-b9e0-d3448ae54df4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a18e40060604986a82e5e356ab42ac1', 'request_id': 'req-47e86861-ea77-4fa6-b9e0-d3448ae54df4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:19.002676 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-47e86861-ea77-4fa6-b9e0-d3448ae54df4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a18e40060604986a82e5e356ab42ac1', 'request_id': 'req-47e86861-ea77-4fa6-b9e0-d3448ae54df4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:19.019214 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-47e86861-ea77-4fa6-b9e0-d3448ae54df4 admin admin] The token cache is being invalidate because project 3a18e40060604986a82e5e356ab42ac1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:57:19.019665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-47e86861-ea77-4fa6-b9e0-d3448ae54df4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:19.020780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-47e86861-ea77-4fa6-b9e0-d3448ae54df4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:19.022620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1342/2713] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:57:18 2022] DELETE /identity/v3/projects/3a18e40060604986a82e5e356ab42ac1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:19.027479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad6207-ae4e-47b4-b7f0-6c4258abfce1 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:19.045782 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad6207-ae4e-47b4-b7f0-6c4258abfce1 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:19.099986 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:19.145924 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b06330-fbe3-4b25-8650-a04c65879fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:19.147194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b06330-fbe3-4b25-8650-a04c65879fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:19.147534 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b06330-fbe3-4b25-8650-a04c65879fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:19.147854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b06330-fbe3-4b25-8650-a04c65879fb9 admin admin] PATH_INFO: `/v3/users/ea205849757643dd9628551a3877050e` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:19.149359 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b06330-fbe3-4b25-8650-a04c65879fb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea205849757643dd9628551a3877050e)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:19.152064 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b06330-fbe3-4b25-8650-a04c65879fb9 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:19.211769 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-64b06330-fbe3-4b25-8650-a04c65879fb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea205849757643dd9628551a3877050e'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:19.230892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-64b06330-fbe3-4b25-8650-a04c65879fb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea205849757643dd9628551a3877050e'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:19.232135 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-64b06330-fbe3-4b25-8650-a04c65879fb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea205849757643dd9628551a3877050e'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:19.246982 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-64b06330-fbe3-4b25-8650-a04c65879fb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea205849757643dd9628551a3877050e'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:19.260263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-64b06330-fbe3-4b25-8650-a04c65879fb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea205849757643dd9628551a3877050e'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:19.267461 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1372/2714] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:57:19 2022] DELETE /identity/v3/users/ea205849757643dd9628551a3877050e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:19.621891 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e86861-ea77-4fa6-b9e0-d3448ae54df4 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:19.623662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-47e86861-ea77-4fa6-b9e0-d3448ae54df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:19.646430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e86861-ea77-4fa6-b9e0-d3448ae54df4 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:19.689595 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:19.720261 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced857c9-40b5-4027-82bf-4fe64971ba51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:19.721156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced857c9-40b5-4027-82bf-4fe64971ba51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:19.721437 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced857c9-40b5-4027-82bf-4fe64971ba51 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:19.721744 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced857c9-40b5-4027-82bf-4fe64971ba51 admin admin] PATH_INFO: `/v3/projects/f51c227d542d4b50ba7a2202f37acead` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:19.723703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced857c9-40b5-4027-82bf-4fe64971ba51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f51c227d542d4b50ba7a2202f37acead)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:19.725994 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced857c9-40b5-4027-82bf-4fe64971ba51 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:19.782018 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ced857c9-40b5-4027-82bf-4fe64971ba51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f51c227d542d4b50ba7a2202f37acead', 'request_id': 'req-ced857c9-40b5-4027-82bf-4fe64971ba51'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:19.790555 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ced857c9-40b5-4027-82bf-4fe64971ba51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f51c227d542d4b50ba7a2202f37acead', 'request_id': 'req-ced857c9-40b5-4027-82bf-4fe64971ba51'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:19.798853 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ced857c9-40b5-4027-82bf-4fe64971ba51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f51c227d542d4b50ba7a2202f37acead', 'request_id': 'req-ced857c9-40b5-4027-82bf-4fe64971ba51'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:19.819011 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ced857c9-40b5-4027-82bf-4fe64971ba51 admin admin] The token cache is being invalidate because project f51c227d542d4b50ba7a2202f37acead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:57:19.819520 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ced857c9-40b5-4027-82bf-4fe64971ba51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:19.820755 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-ced857c9-40b5-4027-82bf-4fe64971ba51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:19.823447 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1343/2715] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:57:19 2022] DELETE /identity/v3/projects/f51c227d542d4b50ba7a2202f37acead => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:19.828587 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28316f50-448d-44d5-80a4-2977bf9bb573 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:19.829022 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28316f50-448d-44d5-80a4-2977bf9bb573 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:19.829297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28316f50-448d-44d5-80a4-2977bf9bb573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:19.851096 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-28316f50-448d-44d5-80a4-2977bf9bb573 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:57:19.852347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-28316f50-448d-44d5-80a4-2977bf9bb573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:19.930753 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1373/2716] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:57:19.944684 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33394d-ba28-4dbe-8c0a-a4838bdc6c9b None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:19.945353 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33394d-ba28-4dbe-8c0a-a4838bdc6c9b None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:19.945353 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33394d-ba28-4dbe-8c0a-a4838bdc6c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:19.966855 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-2f33394d-ba28-4dbe-8c0a-a4838bdc6c9b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:57:19.968203 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-2f33394d-ba28-4dbe-8c0a-a4838bdc6c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:19.977069 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1344/2717] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:57:19.981969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28316f50-448d-44d5-80a4-2977bf9bb573 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:19.993320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28316f50-448d-44d5-80a4-2977bf9bb573 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:19.998555 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c411c-5207-43b2-bd63-8a87b345f8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:19.999239 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c411c-5207-43b2-bd63-8a87b345f8a9 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:19.999528 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c411c-5207-43b2-bd63-8a87b345f8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:19.999810 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c411c-5207-43b2-bd63-8a87b345f8a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:20.000570 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c411c-5207-43b2-bd63-8a87b345f8a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:57:20.000883 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c411c-5207-43b2-bd63-8a87b345f8a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:20.002582 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c411c-5207-43b2-bd63-8a87b345f8a9 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:20.017765 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1374/2718] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:57:20.023917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33394d-ba28-4dbe-8c0a-a4838bdc6c9b None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:20.038446 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33394d-ba28-4dbe-8c0a-a4838bdc6c9b None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:20.043374 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da4170-0cc0-401a-908a-e781da1ca5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:20.044055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da4170-0cc0-401a-908a-e781da1ca5b0 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:20.044342 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da4170-0cc0-401a-908a-e781da1ca5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:20.044624 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da4170-0cc0-401a-908a-e781da1ca5b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:20.045671 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da4170-0cc0-401a-908a-e781da1ca5b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:20.047613 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da4170-0cc0-401a-908a-e781da1ca5b0 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:20.092722 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1345/2719] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:57:20 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:57:20.098133 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c411c-5207-43b2-bd63-8a87b345f8a9 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:20.111153 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c411c-5207-43b2-bd63-8a87b345f8a9 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:20.155911 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:20.188820 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef153481-d194-4082-b5fc-262284a10c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:20.189522 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef153481-d194-4082-b5fc-262284a10c7a admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:20.189847 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef153481-d194-4082-b5fc-262284a10c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:20.190127 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef153481-d194-4082-b5fc-262284a10c7a admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:20.190846 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef153481-d194-4082-b5fc-262284a10c7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:20.193836 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef153481-d194-4082-b5fc-262284a10c7a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:20.217558 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1375/2720] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:57:20 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:57:20.222257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da4170-0cc0-401a-908a-e781da1ca5b0 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:20.234371 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da4170-0cc0-401a-908a-e781da1ca5b0 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:20.240695 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697c5b7-316f-4a02-94c3-9a87accb1737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:20.241510 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697c5b7-316f-4a02-94c3-9a87accb1737 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:20.241939 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697c5b7-316f-4a02-94c3-9a87accb1737 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:20.242318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697c5b7-316f-4a02-94c3-9a87accb1737 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:20.243158 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697c5b7-316f-4a02-94c3-9a87accb1737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:20.244916 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697c5b7-316f-4a02-94c3-9a87accb1737 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:20.257797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1346/2721] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:57:20.262224 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef153481-d194-4082-b5fc-262284a10c7a admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:20.273278 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef153481-d194-4082-b5fc-262284a10c7a admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:20.278754 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42a639-6583-47fc-8af5-aee349cd907f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:20.279515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42a639-6583-47fc-8af5-aee349cd907f admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:20.279892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42a639-6583-47fc-8af5-aee349cd907f admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:20.280251 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42a639-6583-47fc-8af5-aee349cd907f admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:20.281050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42a639-6583-47fc-8af5-aee349cd907f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:20.283209 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42a639-6583-47fc-8af5-aee349cd907f admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:20.294353 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1376/2722] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:57:20.299303 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697c5b7-316f-4a02-94c3-9a87accb1737 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:20.313367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697c5b7-316f-4a02-94c3-9a87accb1737 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:20.319724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b6a78-e7f4-4561-8213-6421a88c6164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:20.320847 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b6a78-e7f4-4561-8213-6421a88c6164 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:20.321237 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b6a78-e7f4-4561-8213-6421a88c6164 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:20.321627 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b6a78-e7f4-4561-8213-6421a88c6164 admin admin] PATH_INFO: `/v3/projects/4069c6002f5e4145acf72bb4f0e035b4/users/59c61a6310344f3d8f2170d167d6312a/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:20.339577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b6a78-e7f4-4561-8213-6421a88c6164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4069c6002f5e4145acf72bb4f0e035b4, user_id=59c61a6310344f3d8f2170d167d6312a, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:20.342633 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b6a78-e7f4-4561-8213-6421a88c6164 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:20.359275 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1347/2723] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:57:20 2022] PUT /identity/v3/projects/4069c6002f5e4145acf72bb4f0e035b4/users/59c61a6310344f3d8f2170d167d6312a/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:20.366240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339387bf-5699-437f-a284-87e8e588994e None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:20.366662 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339387bf-5699-437f-a284-87e8e588994e None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:20.367061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339387bf-5699-437f-a284-87e8e588994e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:20.387296 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-339387bf-5699-437f-a284-87e8e588994e None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:57:20.388546 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-339387bf-5699-437f-a284-87e8e588994e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:20.460597 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1377/2724] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:57:20.471247 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b6a78-e7f4-4561-8213-6421a88c6164 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:20.473977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-bb8b6a78-e7f4-4561-8213-6421a88c6164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:20.546168 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b6a78-e7f4-4561-8213-6421a88c6164 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:20.553212 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e590d1-ed04-4c24-a03f-5c853204db5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:20.554431 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e590d1-ed04-4c24-a03f-5c853204db5e service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:20.554902 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e590d1-ed04-4c24-a03f-5c853204db5e service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:20.555382 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e590d1-ed04-4c24-a03f-5c853204db5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:20.573469 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e590d1-ed04-4c24-a03f-5c853204db5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:20.576401 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e590d1-ed04-4c24-a03f-5c853204db5e service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:20.596004 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1348/2725] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:57:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:57:21.374194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c9eb2-58cb-467e-8e87-1ad850c7cfbe None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:21.375102 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c9eb2-58cb-467e-8e87-1ad850c7cfbe None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:21.375277 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c9eb2-58cb-467e-8e87-1ad850c7cfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:21.427669 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-660c9eb2-58cb-467e-8e87-1ad850c7cfbe None None] MFA Rules not processed for user `59c61a6310344f3d8f2170d167d6312a`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:57:21.430048 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-660c9eb2-58cb-467e-8e87-1ad850c7cfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:21.563581 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1378/2726] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:57:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:57:21.569774 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e590d1-ed04-4c24-a03f-5c853204db5e service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:21.582241 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:21.591622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e590d1-ed04-4c24-a03f-5c853204db5e service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:21.600297 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0097a-1f14-4852-863f-de4331617a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:21.600978 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0097a-1f14-4852-863f-de4331617a0c admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:21.601268 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0097a-1f14-4852-863f-de4331617a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:21.601552 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0097a-1f14-4852-863f-de4331617a0c admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:21.602335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0097a-1f14-4852-863f-de4331617a0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:21.604268 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0097a-1f14-4852-863f-de4331617a0c admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:21.639199 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1349/2727] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:57:21 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:57:21.640432 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c9eb2-58cb-467e-8e87-1ad850c7cfbe None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:21.652968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c9eb2-58cb-467e-8e87-1ad850c7cfbe None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:21.714510 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:21.731907 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7a074-c34c-4b4b-aa92-0f38ab4d4659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:21.732603 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7a074-c34c-4b4b-aa92-0f38ab4d4659 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:21.732888 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7a074-c34c-4b4b-aa92-0f38ab4d4659 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:21.733166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7a074-c34c-4b4b-aa92-0f38ab4d4659 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:21.734035 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7a074-c34c-4b4b-aa92-0f38ab4d4659 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:21.736037 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7a074-c34c-4b4b-aa92-0f38ab4d4659 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:21.758888 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1379/2728] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:57:21 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:57:21.763486 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0097a-1f14-4852-863f-de4331617a0c admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:21.776087 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0097a-1f14-4852-863f-de4331617a0c admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:21.781001 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decf25e-124c-4970-bf4a-dcfbcf1cad0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:21.781861 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decf25e-124c-4970-bf4a-dcfbcf1cad0f admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:21.782146 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decf25e-124c-4970-bf4a-dcfbcf1cad0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:21.782419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decf25e-124c-4970-bf4a-dcfbcf1cad0f admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:21.783163 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2decf25e-124c-4970-bf4a-dcfbcf1cad0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:21.784800 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2decf25e-124c-4970-bf4a-dcfbcf1cad0f admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:21.802534 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1350/2729] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:57:21.807658 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7a074-c34c-4b4b-aa92-0f38ab4d4659 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:21.821435 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7a074-c34c-4b4b-aa92-0f38ab4d4659 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:21.829926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12f6d3-5d3f-4819-a5e3-4dbc38030d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:21.831194 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12f6d3-5d3f-4819-a5e3-4dbc38030d36 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:21.831495 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12f6d3-5d3f-4819-a5e3-4dbc38030d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:21.831822 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12f6d3-5d3f-4819-a5e3-4dbc38030d36 admin admin] PATH_INFO: `/v3/projects/e7f1414f05a94dfcac4292dbdd8a76d4/users/b445dcc9150642128a622e1e3642cb91/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:21.852758 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12f6d3-5d3f-4819-a5e3-4dbc38030d36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7f1414f05a94dfcac4292dbdd8a76d4, user_id=b445dcc9150642128a622e1e3642cb91, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:21.855462 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12f6d3-5d3f-4819-a5e3-4dbc38030d36 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:21.872458 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1380/2730] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:57:21 2022] PUT /identity/v3/projects/e7f1414f05a94dfcac4292dbdd8a76d4/users/b445dcc9150642128a622e1e3642cb91/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:21.877157 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decf25e-124c-4970-bf4a-dcfbcf1cad0f admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:21.889119 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decf25e-124c-4970-bf4a-dcfbcf1cad0f admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:21.965543 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471903a-bd35-4d09-9c2e-9f998a9bf45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:21.966438 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471903a-bd35-4d09-9c2e-9f998a9bf45e admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:21.966822 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471903a-bd35-4d09-9c2e-9f998a9bf45e admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:21.967203 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471903a-bd35-4d09-9c2e-9f998a9bf45e admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:21.968117 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471903a-bd35-4d09-9c2e-9f998a9bf45e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:21.970095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471903a-bd35-4d09-9c2e-9f998a9bf45e admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:21.984215 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1351/2731] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:57:21.991397 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12f6d3-5d3f-4819-a5e3-4dbc38030d36 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:22.003626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12f6d3-5d3f-4819-a5e3-4dbc38030d36 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:22.011769 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0719183-0607-4d4f-8892-41d49ae79b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:22.012813 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0719183-0607-4d4f-8892-41d49ae79b27 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:22.013140 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0719183-0607-4d4f-8892-41d49ae79b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:22.013462 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0719183-0607-4d4f-8892-41d49ae79b27 admin admin] PATH_INFO: `/v3/projects/e7f1414f05a94dfcac4292dbdd8a76d4/users/b445dcc9150642128a622e1e3642cb91/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:22.015892 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0719183-0607-4d4f-8892-41d49ae79b27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7f1414f05a94dfcac4292dbdd8a76d4, user_id=b445dcc9150642128a622e1e3642cb91, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:22.018712 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0719183-0607-4d4f-8892-41d49ae79b27 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:22.039321 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1381/2732] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:57:21 2022] PUT /identity/v3/projects/e7f1414f05a94dfcac4292dbdd8a76d4/users/b445dcc9150642128a622e1e3642cb91/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:22.910637 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d824990-0113-4353-af81-6839c9cf2c67 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:22.911783 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d824990-0113-4353-af81-6839c9cf2c67 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:22.911783 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d824990-0113-4353-af81-6839c9cf2c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:22.943830 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-4d824990-0113-4353-af81-6839c9cf2c67 None None] MFA Rules not processed for user `b445dcc9150642128a622e1e3642cb91`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:57:22.945181 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4d824990-0113-4353-af81-6839c9cf2c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:23.070140 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1352/2733] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:57:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:57:23.083668 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0719183-0607-4d4f-8892-41d49ae79b27 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:23.085436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-f0719183-0607-4d4f-8892-41d49ae79b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:23.155687 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0719183-0607-4d4f-8892-41d49ae79b27 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:23.159626 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e099ec7-5a62-4228-9cd8-11e18093839d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:23.161473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e099ec7-5a62-4228-9cd8-11e18093839d service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:23.161828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e099ec7-5a62-4228-9cd8-11e18093839d service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:23.162111 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e099ec7-5a62-4228-9cd8-11e18093839d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:23.233325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e099ec7-5a62-4228-9cd8-11e18093839d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:23.234937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e099ec7-5a62-4228-9cd8-11e18093839d service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:23.250320 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1382/2734] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:57:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:57:24.689286 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d824990-0113-4353-af81-6839c9cf2c67 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:24.690970 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4d824990-0113-4353-af81-6839c9cf2c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:24.704578 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d824990-0113-4353-af81-6839c9cf2c67 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:24.709149 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395889e3-de4f-44ef-aa81-7ae31d6d422f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:24.709850 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395889e3-de4f-44ef-aa81-7ae31d6d422f service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:24.710132 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395889e3-de4f-44ef-aa81-7ae31d6d422f service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:24.710563 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395889e3-de4f-44ef-aa81-7ae31d6d422f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:24.721242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395889e3-de4f-44ef-aa81-7ae31d6d422f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:24.724029 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395889e3-de4f-44ef-aa81-7ae31d6d422f service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:24.735055 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1353/2735] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:57:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:57:33.242689 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=71,keystone:INSERT=5 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:34.730383 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=81,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:38.955725 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e099ec7-5a62-4228-9cd8-11e18093839d service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:39.047413 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e099ec7-5a62-4228-9cd8-11e18093839d service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:39.056347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190b8e7-695f-4613-8bd5-eba8e6a7a68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:39.057865 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190b8e7-695f-4613-8bd5-eba8e6a7a68b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:39.058359 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190b8e7-695f-4613-8bd5-eba8e6a7a68b admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:39.058956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190b8e7-695f-4613-8bd5-eba8e6a7a68b admin admin] PATH_INFO: `/v3/users/59c61a6310344f3d8f2170d167d6312a` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:39.061241 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190b8e7-695f-4613-8bd5-eba8e6a7a68b admin admin] RBAC: Authorizing `identity:delete_user(user_id=59c61a6310344f3d8f2170d167d6312a)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:39.065166 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e190b8e7-695f-4613-8bd5-eba8e6a7a68b admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:39.158870 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-e190b8e7-695f-4613-8bd5-eba8e6a7a68b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59c61a6310344f3d8f2170d167d6312a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:39.175658 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-e190b8e7-695f-4613-8bd5-eba8e6a7a68b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59c61a6310344f3d8f2170d167d6312a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:39.176471 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-e190b8e7-695f-4613-8bd5-eba8e6a7a68b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59c61a6310344f3d8f2170d167d6312a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:39.186317 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-e190b8e7-695f-4613-8bd5-eba8e6a7a68b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59c61a6310344f3d8f2170d167d6312a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:39.201001 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-e190b8e7-695f-4613-8bd5-eba8e6a7a68b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59c61a6310344f3d8f2170d167d6312a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:39.212492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1383/2736] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:57:38 2022] DELETE /identity/v3/users/59c61a6310344f3d8f2170d167d6312a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:39.456256 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395889e3-de4f-44ef-aa81-7ae31d6d422f service cinder] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:39.472488 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395889e3-de4f-44ef-aa81-7ae31d6d422f service cinder] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:39.576122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cad73-f6f8-41ea-9172-9ec6d0e9b221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:39.577679 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cad73-f6f8-41ea-9172-9ec6d0e9b221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:39.578331 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cad73-f6f8-41ea-9172-9ec6d0e9b221 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:39.578810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cad73-f6f8-41ea-9172-9ec6d0e9b221 admin admin] PATH_INFO: `/v3/projects/4069c6002f5e4145acf72bb4f0e035b4` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:39.582163 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cad73-f6f8-41ea-9172-9ec6d0e9b221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4069c6002f5e4145acf72bb4f0e035b4)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:39.586335 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cad73-f6f8-41ea-9172-9ec6d0e9b221 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:39.643691 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-568cad73-f6f8-41ea-9172-9ec6d0e9b221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4069c6002f5e4145acf72bb4f0e035b4', 'request_id': 'req-568cad73-f6f8-41ea-9172-9ec6d0e9b221'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:39.651457 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-568cad73-f6f8-41ea-9172-9ec6d0e9b221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4069c6002f5e4145acf72bb4f0e035b4', 'request_id': 'req-568cad73-f6f8-41ea-9172-9ec6d0e9b221'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:39.658159 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-568cad73-f6f8-41ea-9172-9ec6d0e9b221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4069c6002f5e4145acf72bb4f0e035b4', 'request_id': 'req-568cad73-f6f8-41ea-9172-9ec6d0e9b221'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:39.674491 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-568cad73-f6f8-41ea-9172-9ec6d0e9b221 admin admin] The token cache is being invalidate because project 4069c6002f5e4145acf72bb4f0e035b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:57:39.674780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-568cad73-f6f8-41ea-9172-9ec6d0e9b221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:39.676483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-568cad73-f6f8-41ea-9172-9ec6d0e9b221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:39.678516 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1354/2737] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:57:39 2022] DELETE /identity/v3/projects/4069c6002f5e4145acf72bb4f0e035b4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:39.684474 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190b8e7-695f-4613-8bd5-eba8e6a7a68b admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:39.686263 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e190b8e7-695f-4613-8bd5-eba8e6a7a68b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:39.701677 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e190b8e7-695f-4613-8bd5-eba8e6a7a68b admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:39.770664 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f308a7f-d7a6-4eb8-b275-565f91d101c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:39.771565 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f308a7f-d7a6-4eb8-b275-565f91d101c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:39.771852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f308a7f-d7a6-4eb8-b275-565f91d101c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:39.772130 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f308a7f-d7a6-4eb8-b275-565f91d101c7 admin admin] PATH_INFO: `/v3/users/b445dcc9150642128a622e1e3642cb91` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:39.773630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f308a7f-d7a6-4eb8-b275-565f91d101c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b445dcc9150642128a622e1e3642cb91)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:39.775918 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f308a7f-d7a6-4eb8-b275-565f91d101c7 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:39.805069 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:39.849792 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9f308a7f-d7a6-4eb8-b275-565f91d101c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b445dcc9150642128a622e1e3642cb91'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:39.872603 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9f308a7f-d7a6-4eb8-b275-565f91d101c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b445dcc9150642128a622e1e3642cb91'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:39.873809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9f308a7f-d7a6-4eb8-b275-565f91d101c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b445dcc9150642128a622e1e3642cb91'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:39.887399 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9f308a7f-d7a6-4eb8-b275-565f91d101c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b445dcc9150642128a622e1e3642cb91'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:39.899439 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9f308a7f-d7a6-4eb8-b275-565f91d101c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b445dcc9150642128a622e1e3642cb91'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:39.906969 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1384/2738] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:57:39 2022] DELETE /identity/v3/users/b445dcc9150642128a622e1e3642cb91 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:40.187366 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cad73-f6f8-41ea-9172-9ec6d0e9b221 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:57:40.189032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-568cad73-f6f8-41ea-9172-9ec6d0e9b221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:57:40.213599 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cad73-f6f8-41ea-9172-9ec6d0e9b221 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:57:40.287203 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea245b8-7280-4898-aacb-49b7a5425095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:57:40.288104 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea245b8-7280-4898-aacb-49b7a5425095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:57:40.288383 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea245b8-7280-4898-aacb-49b7a5425095 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:57:40.288655 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea245b8-7280-4898-aacb-49b7a5425095 admin admin] PATH_INFO: `/v3/projects/e7f1414f05a94dfcac4292dbdd8a76d4` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:57:40.290037 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea245b8-7280-4898-aacb-49b7a5425095 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7f1414f05a94dfcac4292dbdd8a76d4)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:57:40.292066 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea245b8-7280-4898-aacb-49b7a5425095 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:57:40.337479 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:40.354065 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-fea245b8-7280-4898-aacb-49b7a5425095 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7f1414f05a94dfcac4292dbdd8a76d4', 'request_id': 'req-fea245b8-7280-4898-aacb-49b7a5425095'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:40.361566 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-fea245b8-7280-4898-aacb-49b7a5425095 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7f1414f05a94dfcac4292dbdd8a76d4', 'request_id': 'req-fea245b8-7280-4898-aacb-49b7a5425095'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:40.371732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-fea245b8-7280-4898-aacb-49b7a5425095 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7f1414f05a94dfcac4292dbdd8a76d4', 'request_id': 'req-fea245b8-7280-4898-aacb-49b7a5425095'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:40.394085 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-fea245b8-7280-4898-aacb-49b7a5425095 admin admin] The token cache is being invalidate because project e7f1414f05a94dfcac4292dbdd8a76d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:57:40.394467 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-fea245b8-7280-4898-aacb-49b7a5425095 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:40.395772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-fea245b8-7280-4898-aacb-49b7a5425095 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:57:40.397372 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1355/2739] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:57:40 2022] DELETE /identity/v3/projects/e7f1414f05a94dfcac4292dbdd8a76d4 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:57:49.902487 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:57:50.388577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:58:00.229805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce5627-40cd-4531-8dcb-a537869e98d8 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:58:00.230590 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce5627-40cd-4531-8dcb-a537869e98d8 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:58:00.230590 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce5627-40cd-4531-8dcb-a537869e98d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:58:00.249885 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-69ce5627-40cd-4531-8dcb-a537869e98d8 None None] MFA Rules not processed for user `ce59e0072f1c4b91a2825e5fecde7e23`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:58:00.251163 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-69ce5627-40cd-4531-8dcb-a537869e98d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:58:00.341200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1385/2740] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:58:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:58:00.388365 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea245b8-7280-4898-aacb-49b7a5425095 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:58:00.390211 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-fea245b8-7280-4898-aacb-49b7a5425095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:58:00.488174 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea245b8-7280-4898-aacb-49b7a5425095 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:58:00.494559 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17905a67-be65-4582-89b2-a92de8167f1c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493fcde4a45944efbc0b52c44e61b411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:58:00.495717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17905a67-be65-4582-89b2-a92de8167f1c service glance] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:58:00.496226 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17905a67-be65-4582-89b2-a92de8167f1c service glance] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:58:00.496716 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17905a67-be65-4582-89b2-a92de8167f1c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:58:00.500116 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-17905a67-be65-4582-89b2-a92de8167f1c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:58:00.593398 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17905a67-be65-4582-89b2-a92de8167f1c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:58:00.594931 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17905a67-be65-4582-89b2-a92de8167f1c service glance] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:58:00.607691 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1356/2741] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:58:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:58:01.349099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce5627-40cd-4531-8dcb-a537869e98d8 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:58:01.351093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-69ce5627-40cd-4531-8dcb-a537869e98d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:58:01.447840 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce5627-40cd-4531-8dcb-a537869e98d8 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:58:01.451760 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83267bde-da38-453a-b73b-9a8511e9e9cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:58:01.452468 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83267bde-da38-453a-b73b-9a8511e9e9cd service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:58:01.452753 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83267bde-da38-453a-b73b-9a8511e9e9cd service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:58:01.453030 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83267bde-da38-453a-b73b-9a8511e9e9cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:58:01.455050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-83267bde-da38-453a-b73b-9a8511e9e9cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:58:01.472496 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83267bde-da38-453a-b73b-9a8511e9e9cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:58:01.474001 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83267bde-da38-453a-b73b-9a8511e9e9cd service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:58:01.488287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1386/2742] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:58:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:58:10.601945 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=54 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:58:11.481068 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=51 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:58:18.451436 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17905a67-be65-4582-89b2-a92de8167f1c service glance] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:58:18.453572 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-17905a67-be65-4582-89b2-a92de8167f1c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:58:18.476899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17905a67-be65-4582-89b2-a92de8167f1c service glance] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:58:18.482420 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d9ac0-2491-435a-90ea-596ac10f7754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:58:18.483106 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d9ac0-2491-435a-90ea-596ac10f7754 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:58:18.483413 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d9ac0-2491-435a-90ea-596ac10f7754 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:58:18.483668 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d9ac0-2491-435a-90ea-596ac10f7754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:58:18.485896 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-925d9ac0-2491-435a-90ea-596ac10f7754 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:58:18.505611 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d9ac0-2491-435a-90ea-596ac10f7754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:58:18.508363 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d9ac0-2491-435a-90ea-596ac10f7754 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:58:18.525550 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1357/2743] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:58:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:58:24.341219 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83267bde-da38-453a-b73b-9a8511e9e9cd service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:58:24.354589 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83267bde-da38-453a-b73b-9a8511e9e9cd service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:58:24.359218 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00195830-9e07-4d3b-ae3a-385724bd5667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:58:24.360082 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00195830-9e07-4d3b-ae3a-385724bd5667 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:58:24.360483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00195830-9e07-4d3b-ae3a-385724bd5667 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:58:24.360897 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00195830-9e07-4d3b-ae3a-385724bd5667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:58:24.364403 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-00195830-9e07-4d3b-ae3a-385724bd5667 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:58:24.446456 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00195830-9e07-4d3b-ae3a-385724bd5667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:58:24.447945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00195830-9e07-4d3b-ae3a-385724bd5667 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:58:24.460361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1387/2744] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:58:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:58:28.517405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=6 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:58:32.409145 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d9ac0-2491-435a-90ea-596ac10f7754 service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:58:32.410957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-925d9ac0-2491-435a-90ea-596ac10f7754 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:58:32.431665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d9ac0-2491-435a-90ea-596ac10f7754 service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:58:32.439100 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5158d56-f3fb-4c98-b0ba-8e95b7e4f0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:58:32.441769 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5158d56-f3fb-4c98-b0ba-8e95b7e4f0c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:58:32.442063 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5158d56-f3fb-4c98-b0ba-8e95b7e4f0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:58:32.442560 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5158d56-f3fb-4c98-b0ba-8e95b7e4f0c2 admin admin] PATH_INFO: `/v3/users/25024a505319480ab00f4c7e58de600c` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:58:32.444977 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5158d56-f3fb-4c98-b0ba-8e95b7e4f0c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25024a505319480ab00f4c7e58de600c)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:58:32.448927 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5158d56-f3fb-4c98-b0ba-8e95b7e4f0c2 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:58:32.513583 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a5158d56-f3fb-4c98-b0ba-8e95b7e4f0c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25024a505319480ab00f4c7e58de600c'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:32.528807 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a5158d56-f3fb-4c98-b0ba-8e95b7e4f0c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25024a505319480ab00f4c7e58de600c'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:32.538327 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a5158d56-f3fb-4c98-b0ba-8e95b7e4f0c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25024a505319480ab00f4c7e58de600c'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:32.538983 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a5158d56-f3fb-4c98-b0ba-8e95b7e4f0c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25024a505319480ab00f4c7e58de600c'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:32.539620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a5158d56-f3fb-4c98-b0ba-8e95b7e4f0c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25024a505319480ab00f4c7e58de600c'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:32.556169 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1358/2745] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:58:32 2022] DELETE /identity/v3/users/25024a505319480ab00f4c7e58de600c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:58:32.856042 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00195830-9e07-4d3b-ae3a-385724bd5667 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:58:32.875928 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00195830-9e07-4d3b-ae3a-385724bd5667 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:58:32.996301 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517c5f0-d5eb-4505-9dce-17a6ed5b7772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:58:32.997834 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517c5f0-d5eb-4505-9dce-17a6ed5b7772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:58:32.998517 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517c5f0-d5eb-4505-9dce-17a6ed5b7772 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:58:32.998937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517c5f0-d5eb-4505-9dce-17a6ed5b7772 admin admin] PATH_INFO: `/v3/projects/ec9f767e3b4d4fb78f86a611ed9f47c5` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:58:33.001385 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517c5f0-d5eb-4505-9dce-17a6ed5b7772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec9f767e3b4d4fb78f86a611ed9f47c5)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:58:33.005422 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517c5f0-d5eb-4505-9dce-17a6ed5b7772 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:58:33.078471 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9517c5f0-d5eb-4505-9dce-17a6ed5b7772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec9f767e3b4d4fb78f86a611ed9f47c5', 'request_id': 'req-9517c5f0-d5eb-4505-9dce-17a6ed5b7772'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:33.087926 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9517c5f0-d5eb-4505-9dce-17a6ed5b7772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec9f767e3b4d4fb78f86a611ed9f47c5', 'request_id': 'req-9517c5f0-d5eb-4505-9dce-17a6ed5b7772'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:33.098240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9517c5f0-d5eb-4505-9dce-17a6ed5b7772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec9f767e3b4d4fb78f86a611ed9f47c5', 'request_id': 'req-9517c5f0-d5eb-4505-9dce-17a6ed5b7772'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:33.115782 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9517c5f0-d5eb-4505-9dce-17a6ed5b7772 admin admin] The token cache is being invalidate because project ec9f767e3b4d4fb78f86a611ed9f47c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:58:33.116177 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9517c5f0-d5eb-4505-9dce-17a6ed5b7772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:33.117010 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-9517c5f0-d5eb-4505-9dce-17a6ed5b7772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:33.119571 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1388/2746] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:58:32 2022] DELETE /identity/v3/projects/ec9f767e3b4d4fb78f86a611ed9f47c5 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:58:33.124467 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5158d56-f3fb-4c98-b0ba-8e95b7e4f0c2 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:58:33.126296 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a5158d56-f3fb-4c98-b0ba-8e95b7e4f0c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:58:33.151414 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5158d56-f3fb-4c98-b0ba-8e95b7e4f0c2 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:58:33.245042 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60d330-1df0-47f0-8593-2a33d6b84259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:58:33.246067 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60d330-1df0-47f0-8593-2a33d6b84259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:58:33.246348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60d330-1df0-47f0-8593-2a33d6b84259 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:58:33.246622 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60d330-1df0-47f0-8593-2a33d6b84259 admin admin] PATH_INFO: `/v3/users/bf539804a8cf46ada01d8dea4fb1c09f` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:58:33.247913 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60d330-1df0-47f0-8593-2a33d6b84259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf539804a8cf46ada01d8dea4fb1c09f)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:58:33.249971 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60d330-1df0-47f0-8593-2a33d6b84259 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:58:33.309411 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3b60d330-1df0-47f0-8593-2a33d6b84259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf539804a8cf46ada01d8dea4fb1c09f'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:33.325770 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3b60d330-1df0-47f0-8593-2a33d6b84259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf539804a8cf46ada01d8dea4fb1c09f'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:33.335662 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3b60d330-1df0-47f0-8593-2a33d6b84259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf539804a8cf46ada01d8dea4fb1c09f'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:33.336347 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3b60d330-1df0-47f0-8593-2a33d6b84259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf539804a8cf46ada01d8dea4fb1c09f'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:33.337005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-3b60d330-1df0-47f0-8593-2a33d6b84259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf539804a8cf46ada01d8dea4fb1c09f'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:33.339898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:58:33.359171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1359/2747] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:58:33 2022] DELETE /identity/v3/users/bf539804a8cf46ada01d8dea4fb1c09f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:58:33.579003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517c5f0-d5eb-4505-9dce-17a6ed5b7772 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:58:33.581351 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-9517c5f0-d5eb-4505-9dce-17a6ed5b7772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:58:33.602638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517c5f0-d5eb-4505-9dce-17a6ed5b7772 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:58:33.634863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:58:33.686213 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf0691-1c23-4711-ac11-fbbf195eca20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:58:33.686970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf0691-1c23-4711-ac11-fbbf195eca20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:58:33.687256 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf0691-1c23-4711-ac11-fbbf195eca20 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:58:33.687537 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf0691-1c23-4711-ac11-fbbf195eca20 admin admin] PATH_INFO: `/v3/projects/bf5fba05fb7a409c913bab265a982067` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:58:33.688802 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf0691-1c23-4711-ac11-fbbf195eca20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf5fba05fb7a409c913bab265a982067)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:58:33.691572 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf0691-1c23-4711-ac11-fbbf195eca20 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:58:33.747563 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-7edf0691-1c23-4711-ac11-fbbf195eca20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf5fba05fb7a409c913bab265a982067', 'request_id': 'req-7edf0691-1c23-4711-ac11-fbbf195eca20'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:33.756733 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-7edf0691-1c23-4711-ac11-fbbf195eca20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf5fba05fb7a409c913bab265a982067', 'request_id': 'req-7edf0691-1c23-4711-ac11-fbbf195eca20'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:33.766640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-7edf0691-1c23-4711-ac11-fbbf195eca20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf5fba05fb7a409c913bab265a982067', 'request_id': 'req-7edf0691-1c23-4711-ac11-fbbf195eca20'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:33.780226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-7edf0691-1c23-4711-ac11-fbbf195eca20 admin admin] The token cache is being invalidate because project bf5fba05fb7a409c913bab265a982067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:58:33.780596 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-7edf0691-1c23-4711-ac11-fbbf195eca20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:33.781380 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-7edf0691-1c23-4711-ac11-fbbf195eca20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:58:33.783275 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1389/2748] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:58:33 2022] DELETE /identity/v3/projects/bf5fba05fb7a409c913bab265a982067 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:58:43.357061 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=4 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:58:43.776610 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:00.391507 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60d330-1df0-47f0-8593-2a33d6b84259 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:00.394725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-3b60d330-1df0-47f0-8593-2a33d6b84259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:00.542558 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60d330-1df0-47f0-8593-2a33d6b84259 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:00.663223 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965a92c-e9ce-4ff9-a42f-51d9daecd8d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:00.664273 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965a92c-e9ce-4ff9-a42f-51d9daecd8d2 service nova] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:00.664765 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965a92c-e9ce-4ff9-a42f-51d9daecd8d2 service nova] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:00.665257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965a92c-e9ce-4ff9-a42f-51d9daecd8d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:00.669093 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f965a92c-e9ce-4ff9-a42f-51d9daecd8d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:00.692765 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965a92c-e9ce-4ff9-a42f-51d9daecd8d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:00.695816 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965a92c-e9ce-4ff9-a42f-51d9daecd8d2 service nova] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:00.716601 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1360/2749] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:59:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:59:06.597361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf0691-1c23-4711-ac11-fbbf195eca20 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:06.599087 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-7edf0691-1c23-4711-ac11-fbbf195eca20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:06.677510 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf0691-1c23-4711-ac11-fbbf195eca20 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:06.682924 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923839f-cbe3-40db-bf7d-2be6e0363a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:06.683915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923839f-cbe3-40db-bf7d-2be6e0363a19 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:06.684352 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923839f-cbe3-40db-bf7d-2be6e0363a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:06.684752 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923839f-cbe3-40db-bf7d-2be6e0363a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:06.687436 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-d923839f-cbe3-40db-bf7d-2be6e0363a19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:06.759964 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923839f-cbe3-40db-bf7d-2be6e0363a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:06.761327 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923839f-cbe3-40db-bf7d-2be6e0363a19 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:06.772307 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1390/2750] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:59:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:59:10.272683 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965a92c-e9ce-4ff9-a42f-51d9daecd8d2 service nova] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:10.275041 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f965a92c-e9ce-4ff9-a42f-51d9daecd8d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:10.372358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923839f-cbe3-40db-bf7d-2be6e0363a19 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:10.375428 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-d923839f-cbe3-40db-bf7d-2be6e0363a19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:10.386639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965a92c-e9ce-4ff9-a42f-51d9daecd8d2 service nova] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:10.393194 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c867e6-c113-4755-b782-eb033f8b51ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffafb68f1bb941588a5217ad83ad907c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:10.393931 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c867e6-c113-4755-b782-eb033f8b51ba service placement] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:10.394217 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c867e6-c113-4755-b782-eb033f8b51ba service placement] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:10.394499 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c867e6-c113-4755-b782-eb033f8b51ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:10.396350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e1c867e6-c113-4755-b782-eb033f8b51ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:10.402121 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923839f-cbe3-40db-bf7d-2be6e0363a19 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:10.409742 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c867e6-c113-4755-b782-eb033f8b51ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:10.410374 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb26a5-63c9-4508-98f9-176cc212c1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:10.411517 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c867e6-c113-4755-b782-eb033f8b51ba service placement] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:10.411909 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb26a5-63c9-4508-98f9-176cc212c1af admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:10.412426 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb26a5-63c9-4508-98f9-176cc212c1af admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:10.412953 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb26a5-63c9-4508-98f9-176cc212c1af admin admin] PATH_INFO: `/v3/users/ce59e0072f1c4b91a2825e5fecde7e23` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:10.415156 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb26a5-63c9-4508-98f9-176cc212c1af admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce59e0072f1c4b91a2825e5fecde7e23)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:10.419080 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb26a5-63c9-4508-98f9-176cc212c1af admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:10.426282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1361/2751] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:59:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:59:10.491714 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-93eb26a5-63c9-4508-98f9-176cc212c1af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce59e0072f1c4b91a2825e5fecde7e23'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:10.504303 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-93eb26a5-63c9-4508-98f9-176cc212c1af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce59e0072f1c4b91a2825e5fecde7e23'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:10.505068 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-93eb26a5-63c9-4508-98f9-176cc212c1af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce59e0072f1c4b91a2825e5fecde7e23'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:10.514927 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-93eb26a5-63c9-4508-98f9-176cc212c1af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce59e0072f1c4b91a2825e5fecde7e23'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:10.523121 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-93eb26a5-63c9-4508-98f9-176cc212c1af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce59e0072f1c4b91a2825e5fecde7e23'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:10.530268 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1391/2752] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:59:10 2022] DELETE /identity/v3/users/ce59e0072f1c4b91a2825e5fecde7e23 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:10.719697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c867e6-c113-4755-b782-eb033f8b51ba service placement] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:10.732986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c867e6-c113-4755-b782-eb033f8b51ba service placement] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:10.746128 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:10.807314 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719d860-bbc2-4923-a4a0-4369d9cff56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:10.808257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719d860-bbc2-4923-a4a0-4369d9cff56c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:10.808717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719d860-bbc2-4923-a4a0-4369d9cff56c admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:10.809049 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719d860-bbc2-4923-a4a0-4369d9cff56c admin admin] PATH_INFO: `/v3/projects/90d9fb4d2e024e6eac7eeee72e63e904` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:10.810415 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719d860-bbc2-4923-a4a0-4369d9cff56c admin admin] RBAC: Authorizing `identity:delete_project(project_id=90d9fb4d2e024e6eac7eeee72e63e904)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:10.812442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719d860-bbc2-4923-a4a0-4369d9cff56c admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:10.878430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-5719d860-bbc2-4923-a4a0-4369d9cff56c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90d9fb4d2e024e6eac7eeee72e63e904', 'request_id': 'req-5719d860-bbc2-4923-a4a0-4369d9cff56c'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:10.887126 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-5719d860-bbc2-4923-a4a0-4369d9cff56c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90d9fb4d2e024e6eac7eeee72e63e904', 'request_id': 'req-5719d860-bbc2-4923-a4a0-4369d9cff56c'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:10.893815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-5719d860-bbc2-4923-a4a0-4369d9cff56c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90d9fb4d2e024e6eac7eeee72e63e904', 'request_id': 'req-5719d860-bbc2-4923-a4a0-4369d9cff56c'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:10.907952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-5719d860-bbc2-4923-a4a0-4369d9cff56c admin admin] The token cache is being invalidate because project 90d9fb4d2e024e6eac7eeee72e63e904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:59:10.908250 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-5719d860-bbc2-4923-a4a0-4369d9cff56c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:10.909025 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-5719d860-bbc2-4923-a4a0-4369d9cff56c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:10.910638 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1362/2753] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:59:10 2022] DELETE /identity/v3/projects/90d9fb4d2e024e6eac7eeee72e63e904 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:10.915911 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161be37-2c27-4673-9fcf-5c30fa4dbb82 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:10.916338 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161be37-2c27-4673-9fcf-5c30fa4dbb82 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:10.916533 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161be37-2c27-4673-9fcf-5c30fa4dbb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:10.939604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-5161be37-2c27-4673-9fcf-5c30fa4dbb82 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:10.940874 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-5161be37-2c27-4673-9fcf-5c30fa4dbb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:10.959816 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:11.010773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1392/2754] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:59:11.019301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ce3d5-0d4e-408e-aa26-e7aab889f2e0 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:11.019588 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ce3d5-0d4e-408e-aa26-e7aab889f2e0 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:11.019877 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ce3d5-0d4e-408e-aa26-e7aab889f2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:11.039588 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-686ce3d5-0d4e-408e-aa26-e7aab889f2e0 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:11.040836 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-686ce3d5-0d4e-408e-aa26-e7aab889f2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:11.053251 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1363/2755] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:59:11.058696 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5161be37-2c27-4673-9fcf-5c30fa4dbb82 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:11.071077 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5161be37-2c27-4673-9fcf-5c30fa4dbb82 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:11.076372 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046fba1-d26d-4e64-b0b9-05f2d61ea0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:11.077067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046fba1-d26d-4e64-b0b9-05f2d61ea0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:11.077352 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046fba1-d26d-4e64-b0b9-05f2d61ea0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:11.077700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046fba1-d26d-4e64-b0b9-05f2d61ea0c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:11.078427 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046fba1-d26d-4e64-b0b9-05f2d61ea0c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:59:11.078736 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046fba1-d26d-4e64-b0b9-05f2d61ea0c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:11.080389 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046fba1-d26d-4e64-b0b9-05f2d61ea0c0 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:11.094199 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1393/2756] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 15:59:11.100389 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ce3d5-0d4e-408e-aa26-e7aab889f2e0 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:11.109342 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ce3d5-0d4e-408e-aa26-e7aab889f2e0 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:11.115575 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab49d98-394d-4cf0-a77a-9f94effa797b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:11.116266 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab49d98-394d-4cf0-a77a-9f94effa797b admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:11.118058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab49d98-394d-4cf0-a77a-9f94effa797b admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:11.118058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab49d98-394d-4cf0-a77a-9f94effa797b admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:11.118058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab49d98-394d-4cf0-a77a-9f94effa797b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:11.119703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab49d98-394d-4cf0-a77a-9f94effa797b admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:11.164098 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1364/2757] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:59:11 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:59:11.168108 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046fba1-d26d-4e64-b0b9-05f2d61ea0c0 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:11.179704 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046fba1-d26d-4e64-b0b9-05f2d61ea0c0 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:11.246961 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f538c8c-e33c-41d7-836f-34dc747c39ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:11.247661 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f538c8c-e33c-41d7-836f-34dc747c39ff admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:11.247950 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f538c8c-e33c-41d7-836f-34dc747c39ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:11.248245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f538c8c-e33c-41d7-836f-34dc747c39ff admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:11.249057 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f538c8c-e33c-41d7-836f-34dc747c39ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:11.252103 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f538c8c-e33c-41d7-836f-34dc747c39ff admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:11.277997 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1394/2758] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:59:11 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:59:11.279661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab49d98-394d-4cf0-a77a-9f94effa797b admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:11.290518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab49d98-394d-4cf0-a77a-9f94effa797b admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:11.296062 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c02945-8a0b-4933-8e87-67254182706f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:11.296747 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c02945-8a0b-4933-8e87-67254182706f admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:11.297037 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c02945-8a0b-4933-8e87-67254182706f admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:11.297318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c02945-8a0b-4933-8e87-67254182706f admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:11.298254 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c02945-8a0b-4933-8e87-67254182706f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:11.299956 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c02945-8a0b-4933-8e87-67254182706f admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:11.311464 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1365/2759] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:59:11.316308 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f538c8c-e33c-41d7-836f-34dc747c39ff admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:11.327036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f538c8c-e33c-41d7-836f-34dc747c39ff admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:11.332875 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930bcf2-4480-45d1-8b2f-d65207e24572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:11.333612 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930bcf2-4480-45d1-8b2f-d65207e24572 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:11.333922 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930bcf2-4480-45d1-8b2f-d65207e24572 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:11.334201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930bcf2-4480-45d1-8b2f-d65207e24572 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:11.334940 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930bcf2-4480-45d1-8b2f-d65207e24572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:11.336553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930bcf2-4480-45d1-8b2f-d65207e24572 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:11.348339 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1395/2760] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:59:11.352783 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c02945-8a0b-4933-8e87-67254182706f admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:11.363512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c02945-8a0b-4933-8e87-67254182706f admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:11.368325 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b552cc8-3568-44da-8ba3-571902bddd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:11.369225 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b552cc8-3568-44da-8ba3-571902bddd8a admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:11.369511 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b552cc8-3568-44da-8ba3-571902bddd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:11.369828 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b552cc8-3568-44da-8ba3-571902bddd8a admin admin] PATH_INFO: `/v3/projects/fcd9103dd5db493fa1bfbd48344413f3/users/579cf150c6964da4a456f996877825c0/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:11.388885 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b552cc8-3568-44da-8ba3-571902bddd8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcd9103dd5db493fa1bfbd48344413f3, user_id=579cf150c6964da4a456f996877825c0, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:11.391516 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b552cc8-3568-44da-8ba3-571902bddd8a admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:11.407895 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1366/2761] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:59:11 2022] PUT /identity/v3/projects/fcd9103dd5db493fa1bfbd48344413f3/users/579cf150c6964da4a456f996877825c0/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:11.418151 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debc276-6115-4c77-aa07-ca8e2b5bb25f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:11.418756 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debc276-6115-4c77-aa07-ca8e2b5bb25f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:11.418756 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debc276-6115-4c77-aa07-ca8e2b5bb25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:11.448415 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-0debc276-6115-4c77-aa07-ca8e2b5bb25f None None] MFA Rules not processed for user `579cf150c6964da4a456f996877825c0`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:11.449667 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-0debc276-6115-4c77-aa07-ca8e2b5bb25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:11.526380 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1396/2762] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:59:11.531654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b552cc8-3568-44da-8ba3-571902bddd8a admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:11.580430 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:11.606808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b552cc8-3568-44da-8ba3-571902bddd8a admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:11.612483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d9e9d-90c5-4e47-bf5d-7aaed2dd5c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:11.613151 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d9e9d-90c5-4e47-bf5d-7aaed2dd5c57 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:11.613468 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d9e9d-90c5-4e47-bf5d-7aaed2dd5c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:11.613842 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d9e9d-90c5-4e47-bf5d-7aaed2dd5c57 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:11.614548 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d9e9d-90c5-4e47-bf5d-7aaed2dd5c57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:11.616453 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d9e9d-90c5-4e47-bf5d-7aaed2dd5c57 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:11.643556 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1367/2763] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:59:11 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:59:11.647622 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0debc276-6115-4c77-aa07-ca8e2b5bb25f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:11.659396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0debc276-6115-4c77-aa07-ca8e2b5bb25f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:11.674879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:11.729884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd655ea1-f24b-4e96-886e-ea477e0e85a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:11.730979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd655ea1-f24b-4e96-886e-ea477e0e85a2 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:11.731270 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd655ea1-f24b-4e96-886e-ea477e0e85a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:11.731648 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd655ea1-f24b-4e96-886e-ea477e0e85a2 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:11.732889 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd655ea1-f24b-4e96-886e-ea477e0e85a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:11.736045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd655ea1-f24b-4e96-886e-ea477e0e85a2 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:11.767258 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1397/2764] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:59:11 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:59:11.773053 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d9e9d-90c5-4e47-bf5d-7aaed2dd5c57 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:11.786669 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d9e9d-90c5-4e47-bf5d-7aaed2dd5c57 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:11.793035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e88eda-17c0-4f71-ac2a-287321664b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:11.793750 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e88eda-17c0-4f71-ac2a-287321664b7e admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:11.794036 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e88eda-17c0-4f71-ac2a-287321664b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:11.794332 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e88eda-17c0-4f71-ac2a-287321664b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:11.795110 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e88eda-17c0-4f71-ac2a-287321664b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:11.797040 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e88eda-17c0-4f71-ac2a-287321664b7e admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:11.809934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1368/2765] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:59:11.817426 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd655ea1-f24b-4e96-886e-ea477e0e85a2 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:11.836129 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd655ea1-f24b-4e96-886e-ea477e0e85a2 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:11.844133 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956f8d8-c17b-4f19-ad8e-3f827a9d5e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:11.846587 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956f8d8-c17b-4f19-ad8e-3f827a9d5e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:11.847075 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956f8d8-c17b-4f19-ad8e-3f827a9d5e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:11.847565 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956f8d8-c17b-4f19-ad8e-3f827a9d5e6f admin admin] PATH_INFO: `/v3/projects/aea3c51a1d52441291d8fd379680f97f/users/01037eb7f8404a54aedb48407ff65530/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:11.875705 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956f8d8-c17b-4f19-ad8e-3f827a9d5e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea3c51a1d52441291d8fd379680f97f, user_id=01037eb7f8404a54aedb48407ff65530, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:11.881114 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956f8d8-c17b-4f19-ad8e-3f827a9d5e6f admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:11.906592 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1398/2766] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:59:11 2022] PUT /identity/v3/projects/aea3c51a1d52441291d8fd379680f97f/users/01037eb7f8404a54aedb48407ff65530/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:11.912942 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e88eda-17c0-4f71-ac2a-287321664b7e admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:11.924433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e88eda-17c0-4f71-ac2a-287321664b7e admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:11.988038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a67ef8-19b1-4ce4-91a3-c854f0ecd82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:11.988725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a67ef8-19b1-4ce4-91a3-c854f0ecd82f admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:11.989004 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a67ef8-19b1-4ce4-91a3-c854f0ecd82f admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:11.989286 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a67ef8-19b1-4ce4-91a3-c854f0ecd82f admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:11.990051 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a67ef8-19b1-4ce4-91a3-c854f0ecd82f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:11.991921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a67ef8-19b1-4ce4-91a3-c854f0ecd82f admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:12.005396 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1369/2767] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:59:12.010314 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956f8d8-c17b-4f19-ad8e-3f827a9d5e6f admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:12.022786 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956f8d8-c17b-4f19-ad8e-3f827a9d5e6f admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:12.028281 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d226e8-95d5-42fb-87d3-6b2719bcfe03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:12.029236 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d226e8-95d5-42fb-87d3-6b2719bcfe03 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:12.029520 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d226e8-95d5-42fb-87d3-6b2719bcfe03 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:12.029842 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d226e8-95d5-42fb-87d3-6b2719bcfe03 admin admin] PATH_INFO: `/v3/projects/aea3c51a1d52441291d8fd379680f97f/users/01037eb7f8404a54aedb48407ff65530/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:12.031979 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d226e8-95d5-42fb-87d3-6b2719bcfe03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea3c51a1d52441291d8fd379680f97f, user_id=01037eb7f8404a54aedb48407ff65530, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:12.035168 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d226e8-95d5-42fb-87d3-6b2719bcfe03 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:12.047443 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1399/2768] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:59:12 2022] PUT /identity/v3/projects/aea3c51a1d52441291d8fd379680f97f/users/01037eb7f8404a54aedb48407ff65530/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:12.060299 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58ff59-ac48-455c-af3c-7375d2be964c None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:12.060590 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58ff59-ac48-455c-af3c-7375d2be964c None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:12.060875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58ff59-ac48-455c-af3c-7375d2be964c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:12.094308 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-8b58ff59-ac48-455c-af3c-7375d2be964c None None] MFA Rules not processed for user `01037eb7f8404a54aedb48407ff65530`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:12.096403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-8b58ff59-ac48-455c-af3c-7375d2be964c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:12.183344 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1370/2769] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:59:12.196020 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d226e8-95d5-42fb-87d3-6b2719bcfe03 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:12.198068 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-08d226e8-95d5-42fb-87d3-6b2719bcfe03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:12.264406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d226e8-95d5-42fb-87d3-6b2719bcfe03 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:12.268861 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ca566-5831-4181-87f1-2d217b8127d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:12.269524 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ca566-5831-4181-87f1-2d217b8127d7 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:12.269867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ca566-5831-4181-87f1-2d217b8127d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:12.270177 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ca566-5831-4181-87f1-2d217b8127d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:12.334761 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ca566-5831-4181-87f1-2d217b8127d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:12.336257 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ca566-5831-4181-87f1-2d217b8127d7 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:12.347106 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1400/2770] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:59:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 15:59:16.696933 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58ff59-ac48-455c-af3c-7375d2be964c None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:16.698510 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-8b58ff59-ac48-455c-af3c-7375d2be964c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:16.711193 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58ff59-ac48-455c-af3c-7375d2be964c None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:16.714872 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02044fb-b442-45de-b45a-18395677349c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:16.715529 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02044fb-b442-45de-b45a-18395677349c service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:16.715809 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02044fb-b442-45de-b45a-18395677349c service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:16.716085 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02044fb-b442-45de-b45a-18395677349c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:16.726452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02044fb-b442-45de-b45a-18395677349c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:16.727898 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02044fb-b442-45de-b45a-18395677349c service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:16.738587 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1371/2771] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:59:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:59:21.764443 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ca566-5831-4181-87f1-2d217b8127d7 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:21.796672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:21.854890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ca566-5831-4181-87f1-2d217b8127d7 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:21.862684 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34481989-9b1b-4065-9450-9dcf1fd7af78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:21.864258 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34481989-9b1b-4065-9450-9dcf1fd7af78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:21.864782 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34481989-9b1b-4065-9450-9dcf1fd7af78 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:21.865299 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34481989-9b1b-4065-9450-9dcf1fd7af78 admin admin] PATH_INFO: `/v3/users/579cf150c6964da4a456f996877825c0` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:21.867549 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34481989-9b1b-4065-9450-9dcf1fd7af78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=579cf150c6964da4a456f996877825c0)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:21.871424 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34481989-9b1b-4065-9450-9dcf1fd7af78 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:21.940531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-34481989-9b1b-4065-9450-9dcf1fd7af78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '579cf150c6964da4a456f996877825c0'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:21.953246 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-34481989-9b1b-4065-9450-9dcf1fd7af78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '579cf150c6964da4a456f996877825c0'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:21.954122 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-34481989-9b1b-4065-9450-9dcf1fd7af78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '579cf150c6964da4a456f996877825c0'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:21.962790 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-34481989-9b1b-4065-9450-9dcf1fd7af78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '579cf150c6964da4a456f996877825c0'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:21.971518 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-34481989-9b1b-4065-9450-9dcf1fd7af78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '579cf150c6964da4a456f996877825c0'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:21.978111 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1401/2772] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:59:21 2022] DELETE /identity/v3/users/579cf150c6964da4a456f996877825c0 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:21.984298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb8964-2dd4-4d2e-af20-fc6485c1f5c2 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:21.984974 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb8964-2dd4-4d2e-af20-fc6485c1f5c2 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:21.984974 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bb8964-2dd4-4d2e-af20-fc6485c1f5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:22.007305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-a3bb8964-2dd4-4d2e-af20-fc6485c1f5c2 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:22.008439 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a3bb8964-2dd4-4d2e-af20-fc6485c1f5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:22.027311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:22.076780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1372/2773] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:59:22.087159 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34481989-9b1b-4065-9450-9dcf1fd7af78 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:22.088652 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-34481989-9b1b-4065-9450-9dcf1fd7af78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:22.157857 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34481989-9b1b-4065-9450-9dcf1fd7af78 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:22.162453 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3caad-249e-4715-93bf-b7017c59b9e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:22.163175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3caad-249e-4715-93bf-b7017c59b9e6 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:22.163549 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3caad-249e-4715-93bf-b7017c59b9e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:22.163875 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3caad-249e-4715-93bf-b7017c59b9e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:22.175103 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3caad-249e-4715-93bf-b7017c59b9e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:22.176563 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3caad-249e-4715-93bf-b7017c59b9e6 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:22.187249 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1402/2774] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:59:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:59:22.505612 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb8964-2dd4-4d2e-af20-fc6485c1f5c2 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:22.507602 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a3bb8964-2dd4-4d2e-af20-fc6485c1f5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:22.521043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bb8964-2dd4-4d2e-af20-fc6485c1f5c2 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:22.525965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52502fd-6f06-4333-86aa-0c67504f4b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:22.526888 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52502fd-6f06-4333-86aa-0c67504f4b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:22.527193 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52502fd-6f06-4333-86aa-0c67504f4b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:22.527451 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52502fd-6f06-4333-86aa-0c67504f4b17 admin admin] PATH_INFO: `/v3/projects/fcd9103dd5db493fa1bfbd48344413f3` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:22.528791 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52502fd-6f06-4333-86aa-0c67504f4b17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcd9103dd5db493fa1bfbd48344413f3)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:22.530915 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52502fd-6f06-4333-86aa-0c67504f4b17 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:22.589739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f52502fd-6f06-4333-86aa-0c67504f4b17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcd9103dd5db493fa1bfbd48344413f3', 'request_id': 'req-f52502fd-6f06-4333-86aa-0c67504f4b17'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:22.597375 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f52502fd-6f06-4333-86aa-0c67504f4b17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcd9103dd5db493fa1bfbd48344413f3', 'request_id': 'req-f52502fd-6f06-4333-86aa-0c67504f4b17'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:22.603890 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f52502fd-6f06-4333-86aa-0c67504f4b17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcd9103dd5db493fa1bfbd48344413f3', 'request_id': 'req-f52502fd-6f06-4333-86aa-0c67504f4b17'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:22.619918 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f52502fd-6f06-4333-86aa-0c67504f4b17 admin admin] The token cache is being invalidate because project fcd9103dd5db493fa1bfbd48344413f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:59:22.620371 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f52502fd-6f06-4333-86aa-0c67504f4b17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:22.621394 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-f52502fd-6f06-4333-86aa-0c67504f4b17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:22.623393 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1373/2775] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:59:22 2022] DELETE /identity/v3/projects/fcd9103dd5db493fa1bfbd48344413f3 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:22.628497 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3caad-249e-4715-93bf-b7017c59b9e6 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:22.638510 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3caad-249e-4715-93bf-b7017c59b9e6 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:22.692003 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:22.712525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79631a06-05ab-45b1-bd31-64a032cd4a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:22.713947 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79631a06-05ab-45b1-bd31-64a032cd4a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:22.714257 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79631a06-05ab-45b1-bd31-64a032cd4a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:22.714540 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79631a06-05ab-45b1-bd31-64a032cd4a85 admin admin] PATH_INFO: `/v3/users/01037eb7f8404a54aedb48407ff65530` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:22.716065 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79631a06-05ab-45b1-bd31-64a032cd4a85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01037eb7f8404a54aedb48407ff65530)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:22.718086 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79631a06-05ab-45b1-bd31-64a032cd4a85 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:22.773249 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-79631a06-05ab-45b1-bd31-64a032cd4a85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01037eb7f8404a54aedb48407ff65530'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:22.787240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-79631a06-05ab-45b1-bd31-64a032cd4a85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01037eb7f8404a54aedb48407ff65530'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:22.788392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-79631a06-05ab-45b1-bd31-64a032cd4a85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01037eb7f8404a54aedb48407ff65530'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:22.799711 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-79631a06-05ab-45b1-bd31-64a032cd4a85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01037eb7f8404a54aedb48407ff65530'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:22.807955 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-79631a06-05ab-45b1-bd31-64a032cd4a85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01037eb7f8404a54aedb48407ff65530'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:22.814750 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1403/2776] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:59:22 2022] DELETE /identity/v3/users/01037eb7f8404a54aedb48407ff65530 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:23.122416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52502fd-6f06-4333-86aa-0c67504f4b17 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:23.124069 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-f52502fd-6f06-4333-86aa-0c67504f4b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:23.141894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52502fd-6f06-4333-86aa-0c67504f4b17 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:23.220279 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7b077-9212-4046-8685-cd48c728658a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:23.221306 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7b077-9212-4046-8685-cd48c728658a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:23.221618 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7b077-9212-4046-8685-cd48c728658a admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:23.221922 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7b077-9212-4046-8685-cd48c728658a admin admin] PATH_INFO: `/v3/projects/aea3c51a1d52441291d8fd379680f97f` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:23.223318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7b077-9212-4046-8685-cd48c728658a admin admin] RBAC: Authorizing `identity:delete_project(project_id=aea3c51a1d52441291d8fd379680f97f)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:23.225351 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7b077-9212-4046-8685-cd48c728658a admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:23.278986 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:23.280345 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-18f7b077-9212-4046-8685-cd48c728658a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aea3c51a1d52441291d8fd379680f97f', 'request_id': 'req-18f7b077-9212-4046-8685-cd48c728658a'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:23.289105 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-18f7b077-9212-4046-8685-cd48c728658a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aea3c51a1d52441291d8fd379680f97f', 'request_id': 'req-18f7b077-9212-4046-8685-cd48c728658a'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:23.296108 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-18f7b077-9212-4046-8685-cd48c728658a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aea3c51a1d52441291d8fd379680f97f', 'request_id': 'req-18f7b077-9212-4046-8685-cd48c728658a'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:23.310305 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-18f7b077-9212-4046-8685-cd48c728658a admin admin] The token cache is being invalidate because project aea3c51a1d52441291d8fd379680f97f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:59:23.310597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-18f7b077-9212-4046-8685-cd48c728658a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:23.311337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-18f7b077-9212-4046-8685-cd48c728658a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:23.312876 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1374/2777] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:59:23 2022] DELETE /identity/v3/projects/aea3c51a1d52441291d8fd379680f97f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:23.319774 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa724a2d-1144-4130-b440-697c950c0b76 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:23.320050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa724a2d-1144-4130-b440-697c950c0b76 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:23.320331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa724a2d-1144-4130-b440-697c950c0b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:23.340347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-fa724a2d-1144-4130-b440-697c950c0b76 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:23.341623 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-fa724a2d-1144-4130-b440-697c950c0b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:23.413821 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1404/2778] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:59:23.423270 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f3b2e-a712-491c-8c73-1b548123cc14 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:23.424169 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f3b2e-a712-491c-8c73-1b548123cc14 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:23.424169 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f3b2e-a712-491c-8c73-1b548123cc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:23.444754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-e70f3b2e-a712-491c-8c73-1b548123cc14 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:23.446035 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-e70f3b2e-a712-491c-8c73-1b548123cc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:23.455608 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1375/2779] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:59:23.460185 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa724a2d-1144-4130-b440-697c950c0b76 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:23.471104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa724a2d-1144-4130-b440-697c950c0b76 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:23.475514 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170701b-a2f6-4aa3-8177-8ce4b3755907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:23.476179 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170701b-a2f6-4aa3-8177-8ce4b3755907 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:23.476465 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170701b-a2f6-4aa3-8177-8ce4b3755907 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:23.476738 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170701b-a2f6-4aa3-8177-8ce4b3755907 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:23.477479 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170701b-a2f6-4aa3-8177-8ce4b3755907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:59:23.478426 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170701b-a2f6-4aa3-8177-8ce4b3755907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:23.479425 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170701b-a2f6-4aa3-8177-8ce4b3755907 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:23.493085 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1405/2780] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:59:23.499278 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f3b2e-a712-491c-8c73-1b548123cc14 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:23.511629 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f3b2e-a712-491c-8c73-1b548123cc14 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:23.519992 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f84e13-3a3e-446d-8969-e31141580566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:23.521104 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f84e13-3a3e-446d-8969-e31141580566 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:23.521599 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f84e13-3a3e-446d-8969-e31141580566 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:23.522126 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f84e13-3a3e-446d-8969-e31141580566 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:23.523390 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f84e13-3a3e-446d-8969-e31141580566 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:23.527238 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f84e13-3a3e-446d-8969-e31141580566 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:23.560520 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1376/2781] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:59:23 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:59:23.566122 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170701b-a2f6-4aa3-8177-8ce4b3755907 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:23.576860 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170701b-a2f6-4aa3-8177-8ce4b3755907 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:23.647473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:23.665068 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3999f8-8999-40cd-9e49-06984bedacd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:23.666283 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3999f8-8999-40cd-9e49-06984bedacd8 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:23.666854 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3999f8-8999-40cd-9e49-06984bedacd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:23.667150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3999f8-8999-40cd-9e49-06984bedacd8 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:23.667941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3999f8-8999-40cd-9e49-06984bedacd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:23.669967 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3999f8-8999-40cd-9e49-06984bedacd8 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:23.698454 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1406/2782] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:59:23 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:59:23.703049 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f84e13-3a3e-446d-8969-e31141580566 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:23.716459 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f84e13-3a3e-446d-8969-e31141580566 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:23.723044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b45a0-f681-4cfa-9fad-e9b8702be0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:23.724215 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b45a0-f681-4cfa-9fad-e9b8702be0ef admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:23.724734 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b45a0-f681-4cfa-9fad-e9b8702be0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:23.725239 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b45a0-f681-4cfa-9fad-e9b8702be0ef admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:23.726769 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b45a0-f681-4cfa-9fad-e9b8702be0ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:23.729761 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b45a0-f681-4cfa-9fad-e9b8702be0ef admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:23.753474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1377/2783] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:59:23.757291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3999f8-8999-40cd-9e49-06984bedacd8 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:23.770914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3999f8-8999-40cd-9e49-06984bedacd8 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:23.780756 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4bce17-ae1d-460b-bc7c-4547c9bd8ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:23.781581 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bce17-ae1d-460b-bc7c-4547c9bd8ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:23.781753 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bce17-ae1d-460b-bc7c-4547c9bd8ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:23.782618 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bce17-ae1d-460b-bc7c-4547c9bd8ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:23.783404 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4bce17-ae1d-460b-bc7c-4547c9bd8ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:23.785084 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4bce17-ae1d-460b-bc7c-4547c9bd8ba0 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:23.797819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1407/2784] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:59:23.801894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b45a0-f681-4cfa-9fad-e9b8702be0ef admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:23.812365 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b45a0-f681-4cfa-9fad-e9b8702be0ef admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:23.817882 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da28bd-6e19-4e19-a7c2-357723216b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:23.818738 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da28bd-6e19-4e19-a7c2-357723216b89 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:23.819045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da28bd-6e19-4e19-a7c2-357723216b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:23.819317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da28bd-6e19-4e19-a7c2-357723216b89 admin admin] PATH_INFO: `/v3/projects/84d3284ca3b94390ae7692daa47cb400/users/1275c3219b044bc7bfe248fd22ee3404/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:23.836514 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da28bd-6e19-4e19-a7c2-357723216b89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84d3284ca3b94390ae7692daa47cb400, user_id=1275c3219b044bc7bfe248fd22ee3404, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:23.839219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da28bd-6e19-4e19-a7c2-357723216b89 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:23.855257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1378/2785] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:59:23 2022] PUT /identity/v3/projects/84d3284ca3b94390ae7692daa47cb400/users/1275c3219b044bc7bfe248fd22ee3404/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:23.867156 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478860cd-07d7-4b5c-9879-451785e4e72a None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:23.867429 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478860cd-07d7-4b5c-9879-451785e4e72a None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:23.867706 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478860cd-07d7-4b5c-9879-451785e4e72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:23.896768 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-478860cd-07d7-4b5c-9879-451785e4e72a None None] MFA Rules not processed for user `1275c3219b044bc7bfe248fd22ee3404`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:23.898055 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-478860cd-07d7-4b5c-9879-451785e4e72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:23.968573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1408/2786] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:59:23.981032 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da28bd-6e19-4e19-a7c2-357723216b89 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:23.983044 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-b2da28bd-6e19-4e19-a7c2-357723216b89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:24.044234 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da28bd-6e19-4e19-a7c2-357723216b89 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:24.048483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f1397-838e-4f3e-81d6-ab699231528e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '493fcde4a45944efbc0b52c44e61b411', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:24.049134 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f1397-838e-4f3e-81d6-ab699231528e service glance] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:24.049416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f1397-838e-4f3e-81d6-ab699231528e service glance] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:24.049723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f1397-838e-4f3e-81d6-ab699231528e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:24.059470 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f1397-838e-4f3e-81d6-ab699231528e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:24.060939 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f1397-838e-4f3e-81d6-ab699231528e service glance] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:24.076875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1379/2787] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:59:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:59:33.965319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=42,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:34.069233 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=60,keystone:DELETE=3,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:39.769855 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478860cd-07d7-4b5c-9879-451785e4e72a None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:39.849280 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478860cd-07d7-4b5c-9879-451785e4e72a None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:39.856801 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3d9e3-40b4-4027-b1d6-ff8ad63b50f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:39.858383 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3d9e3-40b4-4027-b1d6-ff8ad63b50f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:39.858904 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3d9e3-40b4-4027-b1d6-ff8ad63b50f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:39.859418 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3d9e3-40b4-4027-b1d6-ff8ad63b50f7 admin admin] PATH_INFO: `/v3/users/1275c3219b044bc7bfe248fd22ee3404` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:39.861720 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3d9e3-40b4-4027-b1d6-ff8ad63b50f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1275c3219b044bc7bfe248fd22ee3404)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:39.865204 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3d9e3-40b4-4027-b1d6-ff8ad63b50f7 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:39.939551 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-eae3d9e3-40b4-4027-b1d6-ff8ad63b50f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1275c3219b044bc7bfe248fd22ee3404'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:39.951999 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-eae3d9e3-40b4-4027-b1d6-ff8ad63b50f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1275c3219b044bc7bfe248fd22ee3404'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:39.952831 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-eae3d9e3-40b4-4027-b1d6-ff8ad63b50f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1275c3219b044bc7bfe248fd22ee3404'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:39.963309 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-eae3d9e3-40b4-4027-b1d6-ff8ad63b50f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1275c3219b044bc7bfe248fd22ee3404'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:39.971675 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-eae3d9e3-40b4-4027-b1d6-ff8ad63b50f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1275c3219b044bc7bfe248fd22ee3404'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:39.978527 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1409/2788] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:59:39 2022] DELETE /identity/v3/users/1275c3219b044bc7bfe248fd22ee3404 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:39.983524 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f0bda-da02-4017-a20b-1f07ce5c162c None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:39.984154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f0bda-da02-4017-a20b-1f07ce5c162c None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:39.984154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f0bda-da02-4017-a20b-1f07ce5c162c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:40.002949 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-c78f0bda-da02-4017-a20b-1f07ce5c162c None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:40.004076 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-c78f0bda-da02-4017-a20b-1f07ce5c162c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:40.068456 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1380/2789] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:59:40.083775 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3d9e3-40b4-4027-b1d6-ff8ad63b50f7 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:40.085275 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-eae3d9e3-40b4-4027-b1d6-ff8ad63b50f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:40.150613 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3d9e3-40b4-4027-b1d6-ff8ad63b50f7 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:40.154591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114576f2-b3f8-406d-9268-68c88afceef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:40.155264 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114576f2-b3f8-406d-9268-68c88afceef5 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:40.155553 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114576f2-b3f8-406d-9268-68c88afceef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:40.155832 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114576f2-b3f8-406d-9268-68c88afceef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:40.167023 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114576f2-b3f8-406d-9268-68c88afceef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:40.168489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114576f2-b3f8-406d-9268-68c88afceef5 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:40.179950 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1410/2790] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:59:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:59:40.509391 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f0bda-da02-4017-a20b-1f07ce5c162c None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:40.511359 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-c78f0bda-da02-4017-a20b-1f07ce5c162c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:40.529873 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f0bda-da02-4017-a20b-1f07ce5c162c None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:40.537486 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642b67a-4e86-4e61-b313-e7cdc980da1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:40.538784 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642b67a-4e86-4e61-b313-e7cdc980da1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:40.539124 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642b67a-4e86-4e61-b313-e7cdc980da1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:40.539535 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642b67a-4e86-4e61-b313-e7cdc980da1f admin admin] PATH_INFO: `/v3/projects/84d3284ca3b94390ae7692daa47cb400` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:40.541688 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642b67a-4e86-4e61-b313-e7cdc980da1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=84d3284ca3b94390ae7692daa47cb400)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:40.544620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642b67a-4e86-4e61-b313-e7cdc980da1f admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:40.604266 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2642b67a-4e86-4e61-b313-e7cdc980da1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84d3284ca3b94390ae7692daa47cb400', 'request_id': 'req-2642b67a-4e86-4e61-b313-e7cdc980da1f'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:40.611380 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2642b67a-4e86-4e61-b313-e7cdc980da1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84d3284ca3b94390ae7692daa47cb400', 'request_id': 'req-2642b67a-4e86-4e61-b313-e7cdc980da1f'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:40.617771 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2642b67a-4e86-4e61-b313-e7cdc980da1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84d3284ca3b94390ae7692daa47cb400', 'request_id': 'req-2642b67a-4e86-4e61-b313-e7cdc980da1f'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:40.634010 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2642b67a-4e86-4e61-b313-e7cdc980da1f admin admin] The token cache is being invalidate because project 84d3284ca3b94390ae7692daa47cb400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:59:40.634395 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2642b67a-4e86-4e61-b313-e7cdc980da1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:40.635147 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-2642b67a-4e86-4e61-b313-e7cdc980da1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:40.637166 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1381/2791] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:59:40 2022] DELETE /identity/v3/projects/84d3284ca3b94390ae7692daa47cb400 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:40.643410 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90cab8-a604-4eaa-a416-4385cef9a254 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:40.643771 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90cab8-a604-4eaa-a416-4385cef9a254 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:40.644123 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90cab8-a604-4eaa-a416-4385cef9a254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:40.665131 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-9c90cab8-a604-4eaa-a416-4385cef9a254 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:40.666407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-9c90cab8-a604-4eaa-a416-4385cef9a254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:40.688867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:40.754327 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1411/2792] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:59:40.764252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e69722-d500-48bf-be71-62668c019aa8 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:40.764608 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e69722-d500-48bf-be71-62668c019aa8 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:40.765013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e69722-d500-48bf-be71-62668c019aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:40.789350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-93e69722-d500-48bf-be71-62668c019aa8 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:40.790636 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-93e69722-d500-48bf-be71-62668c019aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:40.800512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1382/2793] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:59:40.805316 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90cab8-a604-4eaa-a416-4385cef9a254 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:40.816972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90cab8-a604-4eaa-a416-4385cef9a254 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:40.823068 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a98de-fca0-4b31-accd-183847ddabf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:40.823863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a98de-fca0-4b31-accd-183847ddabf6 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:40.824176 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a98de-fca0-4b31-accd-183847ddabf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:40.824593 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a98de-fca0-4b31-accd-183847ddabf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:40.825720 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a98de-fca0-4b31-accd-183847ddabf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:59:40.826156 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a98de-fca0-4b31-accd-183847ddabf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:40.827841 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a98de-fca0-4b31-accd-183847ddabf6 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:40.842323 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1412/2794] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:59:40.847656 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e69722-d500-48bf-be71-62668c019aa8 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:40.861954 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e69722-d500-48bf-be71-62668c019aa8 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:40.866959 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ff4d4-e5d1-4cd0-9d4f-b8bdbaf7abca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:40.867797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ff4d4-e5d1-4cd0-9d4f-b8bdbaf7abca admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:40.868088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ff4d4-e5d1-4cd0-9d4f-b8bdbaf7abca admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:40.868389 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ff4d4-e5d1-4cd0-9d4f-b8bdbaf7abca admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:40.869117 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ff4d4-e5d1-4cd0-9d4f-b8bdbaf7abca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:40.871286 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64ff4d4-e5d1-4cd0-9d4f-b8bdbaf7abca admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:40.903237 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1383/2795] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:59:40 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:59:40.910715 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a98de-fca0-4b31-accd-183847ddabf6 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:40.927106 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a98de-fca0-4b31-accd-183847ddabf6 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:40.998137 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8205e-ef8d-484e-99a6-87110e320e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:40.998867 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8205e-ef8d-484e-99a6-87110e320e17 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:40.999146 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8205e-ef8d-484e-99a6-87110e320e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:40.999423 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8205e-ef8d-484e-99a6-87110e320e17 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:41.000302 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8205e-ef8d-484e-99a6-87110e320e17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:41.002648 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8205e-ef8d-484e-99a6-87110e320e17 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:41.032034 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1413/2796] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:59:40 2022] POST /identity/v3/users => generated 289 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:59:41.037897 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ff4d4-e5d1-4cd0-9d4f-b8bdbaf7abca admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:41.050776 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ff4d4-e5d1-4cd0-9d4f-b8bdbaf7abca admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:41.056619 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8749a-ce4d-4ea4-bf68-527d59d83fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:41.057349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8749a-ce4d-4ea4-bf68-527d59d83fec admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:41.057711 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8749a-ce4d-4ea4-bf68-527d59d83fec admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:41.058012 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8749a-ce4d-4ea4-bf68-527d59d83fec admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:41.059014 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8749a-ce4d-4ea4-bf68-527d59d83fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:41.061434 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8749a-ce4d-4ea4-bf68-527d59d83fec admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:41.078167 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1384/2797] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:59:41.083659 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8205e-ef8d-484e-99a6-87110e320e17 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:41.099307 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8205e-ef8d-484e-99a6-87110e320e17 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:41.104213 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7628f04-6961-4d31-814e-76b0a6234550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:41.104901 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7628f04-6961-4d31-814e-76b0a6234550 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:41.105186 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7628f04-6961-4d31-814e-76b0a6234550 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:41.105473 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7628f04-6961-4d31-814e-76b0a6234550 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:41.106308 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7628f04-6961-4d31-814e-76b0a6234550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:41.107923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7628f04-6961-4d31-814e-76b0a6234550 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:41.120630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1414/2798] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:59:41.125123 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8749a-ce4d-4ea4-bf68-527d59d83fec admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:41.136512 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8749a-ce4d-4ea4-bf68-527d59d83fec admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:41.142835 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eecec1b-2b02-4b39-88c6-676a42793cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:41.143811 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecec1b-2b02-4b39-88c6-676a42793cf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:41.144292 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecec1b-2b02-4b39-88c6-676a42793cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:41.144796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eecec1b-2b02-4b39-88c6-676a42793cf5 admin admin] PATH_INFO: `/v3/projects/bf1548c988f94ff9b4bd15a848825365/users/a915b0ed19e84f04a25c5c6345ffb7fa/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:41.171722 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eecec1b-2b02-4b39-88c6-676a42793cf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf1548c988f94ff9b4bd15a848825365, user_id=a915b0ed19e84f04a25c5c6345ffb7fa, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:41.175665 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eecec1b-2b02-4b39-88c6-676a42793cf5 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:41.198434 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1385/2799] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:59:41 2022] PUT /identity/v3/projects/bf1548c988f94ff9b4bd15a848825365/users/a915b0ed19e84f04a25c5c6345ffb7fa/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 15:59:41.210643 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046bebb-d070-4e25-be47-65ba0fcc2a8f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:41.211272 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046bebb-d070-4e25-be47-65ba0fcc2a8f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:41.211272 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046bebb-d070-4e25-be47-65ba0fcc2a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:41.266977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-d046bebb-d070-4e25-be47-65ba0fcc2a8f None None] MFA Rules not processed for user `a915b0ed19e84f04a25c5c6345ffb7fa`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:41.269201 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-d046bebb-d070-4e25-be47-65ba0fcc2a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:41.397529 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1415/2800] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:59:41.403490 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eecec1b-2b02-4b39-88c6-676a42793cf5 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:41.450550 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:41.528204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eecec1b-2b02-4b39-88c6-676a42793cf5 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:41.536248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd471d6f-af73-4bde-9422-77753c5a3c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:41.537427 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd471d6f-af73-4bde-9422-77753c5a3c3c admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:41.537996 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd471d6f-af73-4bde-9422-77753c5a3c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:41.538492 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd471d6f-af73-4bde-9422-77753c5a3c3c admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:41.539784 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd471d6f-af73-4bde-9422-77753c5a3c3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:41.542597 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd471d6f-af73-4bde-9422-77753c5a3c3c admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:41.580095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1386/2801] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:59:41 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:59:41.584441 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046bebb-d070-4e25-be47-65ba0fcc2a8f None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:41.598459 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046bebb-d070-4e25-be47-65ba0fcc2a8f None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:41.615598 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:41.674366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c6f68-93dc-46b5-a69a-ae009a428143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:41.674959 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c6f68-93dc-46b5-a69a-ae009a428143 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:41.675256 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c6f68-93dc-46b5-a69a-ae009a428143 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:41.675531 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c6f68-93dc-46b5-a69a-ae009a428143 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:41.676238 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c6f68-93dc-46b5-a69a-ae009a428143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:41.678226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c6f68-93dc-46b5-a69a-ae009a428143 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:41.707290 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1416/2802] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:59:41 2022] POST /identity/v3/users => generated 289 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:59:41.712382 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd471d6f-af73-4bde-9422-77753c5a3c3c admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:41.725083 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd471d6f-af73-4bde-9422-77753c5a3c3c admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:41.730721 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb76e3-60aa-4969-9e4c-0a16feadd4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:41.731465 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb76e3-60aa-4969-9e4c-0a16feadd4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:41.731813 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb76e3-60aa-4969-9e4c-0a16feadd4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:41.732136 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb76e3-60aa-4969-9e4c-0a16feadd4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:41.732928 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb76e3-60aa-4969-9e4c-0a16feadd4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:41.735784 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb76e3-60aa-4969-9e4c-0a16feadd4e4 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:41.751262 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1387/2803] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:59:41.758028 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c6f68-93dc-46b5-a69a-ae009a428143 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:41.770572 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c6f68-93dc-46b5-a69a-ae009a428143 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:41.776093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70ebd2-786e-4631-aad3-762fd35a8f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:41.776819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70ebd2-786e-4631-aad3-762fd35a8f19 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:41.777106 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70ebd2-786e-4631-aad3-762fd35a8f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:41.777388 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70ebd2-786e-4631-aad3-762fd35a8f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:41.778175 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70ebd2-786e-4631-aad3-762fd35a8f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:41.779811 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70ebd2-786e-4631-aad3-762fd35a8f19 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:41.793700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1417/2804] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:59:41.801979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb76e3-60aa-4969-9e4c-0a16feadd4e4 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:41.814703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb76e3-60aa-4969-9e4c-0a16feadd4e4 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:41.820354 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e060e-dcbc-4d4d-b89d-7d168c59fdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:41.821296 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e060e-dcbc-4d4d-b89d-7d168c59fdcd admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:41.821696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e060e-dcbc-4d4d-b89d-7d168c59fdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:41.821924 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e060e-dcbc-4d4d-b89d-7d168c59fdcd admin admin] PATH_INFO: `/v3/projects/7eb5e4290a0d40c6aee41776c3a34f48/users/c3de986750794d12934319eea0557b44/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:41.841663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e060e-dcbc-4d4d-b89d-7d168c59fdcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eb5e4290a0d40c6aee41776c3a34f48, user_id=c3de986750794d12934319eea0557b44, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:41.844944 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e060e-dcbc-4d4d-b89d-7d168c59fdcd admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:41.860660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1388/2805] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:59:41 2022] PUT /identity/v3/projects/7eb5e4290a0d40c6aee41776c3a34f48/users/c3de986750794d12934319eea0557b44/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:41.873562 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb2f3e-7c0c-41da-b838-cc7a790b6740 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:41.874591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb2f3e-7c0c-41da-b838-cc7a790b6740 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:41.874591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb2f3e-7c0c-41da-b838-cc7a790b6740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:41.912604 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-7ceb2f3e-7c0c-41da-b838-cc7a790b6740 None None] MFA Rules not processed for user `c3de986750794d12934319eea0557b44`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:41.913890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-7ceb2f3e-7c0c-41da-b838-cc7a790b6740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:41.996499 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1418/2806] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:59:42.001833 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e060e-dcbc-4d4d-b89d-7d168c59fdcd admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:42.080904 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e060e-dcbc-4d4d-b89d-7d168c59fdcd admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:42.087213 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98657c95-e1a1-453f-ad3b-58870b4c5daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:42.088095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98657c95-e1a1-453f-ad3b-58870b4c5daf admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:42.088390 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98657c95-e1a1-453f-ad3b-58870b4c5daf admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:42.088674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98657c95-e1a1-453f-ad3b-58870b4c5daf admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:42.089389 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98657c95-e1a1-453f-ad3b-58870b4c5daf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:42.091641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98657c95-e1a1-453f-ad3b-58870b4c5daf admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:42.123219 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1389/2807] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:59:41 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:59:42.128773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb2f3e-7c0c-41da-b838-cc7a790b6740 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:42.144036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb2f3e-7c0c-41da-b838-cc7a790b6740 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:42.211754 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac265e2-090d-464a-b6cb-9e147ceb642d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:42.212452 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac265e2-090d-464a-b6cb-9e147ceb642d admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:42.212740 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac265e2-090d-464a-b6cb-9e147ceb642d admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:42.213112 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac265e2-090d-464a-b6cb-9e147ceb642d admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:42.213774 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac265e2-090d-464a-b6cb-9e147ceb642d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:42.215789 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac265e2-090d-464a-b6cb-9e147ceb642d admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:42.240627 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1419/2808] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:59:42 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 15:59:42.245161 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98657c95-e1a1-453f-ad3b-58870b4c5daf admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:42.258474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98657c95-e1a1-453f-ad3b-58870b4c5daf admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:42.264647 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402bcda-eda2-41c8-9b48-d30616119800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:42.265485 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402bcda-eda2-41c8-9b48-d30616119800 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:42.265965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402bcda-eda2-41c8-9b48-d30616119800 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:42.266358 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402bcda-eda2-41c8-9b48-d30616119800 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:42.267462 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402bcda-eda2-41c8-9b48-d30616119800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:42.269350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402bcda-eda2-41c8-9b48-d30616119800 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:42.283994 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1390/2809] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 15:59:42.288580 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac265e2-090d-464a-b6cb-9e147ceb642d admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:42.302124 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac265e2-090d-464a-b6cb-9e147ceb642d admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:42.307678 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbd68b-5b70-42c0-830b-7bfaa2ede29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:42.308772 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbd68b-5b70-42c0-830b-7bfaa2ede29b admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:42.309161 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbd68b-5b70-42c0-830b-7bfaa2ede29b admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:42.309494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbd68b-5b70-42c0-830b-7bfaa2ede29b admin admin] PATH_INFO: `/v3/projects/fab888eaf583424e87326f2af137a9d3/users/ea89ecf7a3164958918ff93e34e3d477/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:42.330228 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:42.334975 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbbd68b-5b70-42c0-830b-7bfaa2ede29b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab888eaf583424e87326f2af137a9d3, user_id=ea89ecf7a3164958918ff93e34e3d477, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:42.338710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbbd68b-5b70-42c0-830b-7bfaa2ede29b admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:42.359852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1420/2810] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:59:42 2022] PUT /identity/v3/projects/fab888eaf583424e87326f2af137a9d3/users/ea89ecf7a3164958918ff93e34e3d477/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:42.365563 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402bcda-eda2-41c8-9b48-d30616119800 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:42.378370 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402bcda-eda2-41c8-9b48-d30616119800 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:42.433435 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:42.457078 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496443c-6437-4b78-8c89-2ebb9e17295e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:42.458212 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496443c-6437-4b78-8c89-2ebb9e17295e admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:42.458610 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496443c-6437-4b78-8c89-2ebb9e17295e admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:42.459095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496443c-6437-4b78-8c89-2ebb9e17295e admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:42.459955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496443c-6437-4b78-8c89-2ebb9e17295e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:42.461712 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e496443c-6437-4b78-8c89-2ebb9e17295e admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:42.476362 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1391/2811] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:59:42.481172 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbd68b-5b70-42c0-830b-7bfaa2ede29b admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:42.493602 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbd68b-5b70-42c0-830b-7bfaa2ede29b admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:42.498927 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b7384-c558-4fe5-b67e-a0dda29a7c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:42.500009 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b7384-c558-4fe5-b67e-a0dda29a7c33 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:42.501764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b7384-c558-4fe5-b67e-a0dda29a7c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:42.501764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b7384-c558-4fe5-b67e-a0dda29a7c33 admin admin] PATH_INFO: `/v3/projects/fab888eaf583424e87326f2af137a9d3/users/ea89ecf7a3164958918ff93e34e3d477/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:42.503755 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b7384-c558-4fe5-b67e-a0dda29a7c33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab888eaf583424e87326f2af137a9d3, user_id=ea89ecf7a3164958918ff93e34e3d477, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:42.506945 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b7384-c558-4fe5-b67e-a0dda29a7c33 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:42.520251 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1421/2812] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:59:42 2022] PUT /identity/v3/projects/fab888eaf583424e87326f2af137a9d3/users/ea89ecf7a3164958918ff93e34e3d477/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:42.530818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e7280-68f4-42bc-a12d-37218f7fe510 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:42.531091 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e7280-68f4-42bc-a12d-37218f7fe510 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:42.531399 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e7280-68f4-42bc-a12d-37218f7fe510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:42.565405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-930e7280-68f4-42bc-a12d-37218f7fe510 None None] MFA Rules not processed for user `ea89ecf7a3164958918ff93e34e3d477`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:42.566729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-930e7280-68f4-42bc-a12d-37218f7fe510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:42.647077 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1392/2813] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:59:42.660496 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b7384-c558-4fe5-b67e-a0dda29a7c33 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:42.662362 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-e50b7384-c558-4fe5-b67e-a0dda29a7c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:42.728070 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b7384-c558-4fe5-b67e-a0dda29a7c33 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:42.732114 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe885cf-e6b7-4f58-8d4d-8be9a25316b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:42.732777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe885cf-e6b7-4f58-8d4d-8be9a25316b1 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:42.733066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe885cf-e6b7-4f58-8d4d-8be9a25316b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:42.733347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe885cf-e6b7-4f58-8d4d-8be9a25316b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:42.802780 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe885cf-e6b7-4f58-8d4d-8be9a25316b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:42.804908 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe885cf-e6b7-4f58-8d4d-8be9a25316b1 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:42.816049 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1422/2814] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:59:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:59:44.433357 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e7280-68f4-42bc-a12d-37218f7fe510 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:44.434932 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-930e7280-68f4-42bc-a12d-37218f7fe510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:44.449946 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e7280-68f4-42bc-a12d-37218f7fe510 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:44.454518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd709f-3dc4-43d2-8848-67268d712d33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:44.455346 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd709f-3dc4-43d2-8848-67268d712d33 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:44.455725 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd709f-3dc4-43d2-8848-67268d712d33 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:44.456202 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd709f-3dc4-43d2-8848-67268d712d33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:44.532641 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd709f-3dc4-43d2-8848-67268d712d33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:44.534212 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd709f-3dc4-43d2-8848-67268d712d33 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:44.545012 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1393/2815] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:59:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:59:45.212952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe885cf-e6b7-4f58-8d4d-8be9a25316b1 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:45.223155 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe885cf-e6b7-4f58-8d4d-8be9a25316b1 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:45.227045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec103d-8fc1-4b7b-939b-7e16c7edd8f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:45.227716 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec103d-8fc1-4b7b-939b-7e16c7edd8f2 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:45.228001 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec103d-8fc1-4b7b-939b-7e16c7edd8f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:45.228277 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec103d-8fc1-4b7b-939b-7e16c7edd8f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:45.240185 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec103d-8fc1-4b7b-939b-7e16c7edd8f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:45.241781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec103d-8fc1-4b7b-939b-7e16c7edd8f2 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:45.254292 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1423/2816] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:59:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:59:49.273254 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd709f-3dc4-43d2-8848-67268d712d33 service cinder] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:49.355141 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd709f-3dc4-43d2-8848-67268d712d33 service cinder] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:49.360638 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b3912-f00e-4b04-a5f5-d458b64a7392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:49.361527 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b3912-f00e-4b04-a5f5-d458b64a7392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:49.361841 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b3912-f00e-4b04-a5f5-d458b64a7392 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:49.362115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b3912-f00e-4b04-a5f5-d458b64a7392 admin admin] PATH_INFO: `/v3/users/a915b0ed19e84f04a25c5c6345ffb7fa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:49.363389 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b3912-f00e-4b04-a5f5-d458b64a7392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a915b0ed19e84f04a25c5c6345ffb7fa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:49.365322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b3912-f00e-4b04-a5f5-d458b64a7392 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:49.370601 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=100 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:49.417809 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-865b3912-f00e-4b04-a5f5-d458b64a7392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a915b0ed19e84f04a25c5c6345ffb7fa'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:49.431712 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-865b3912-f00e-4b04-a5f5-d458b64a7392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a915b0ed19e84f04a25c5c6345ffb7fa'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:49.442156 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-865b3912-f00e-4b04-a5f5-d458b64a7392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a915b0ed19e84f04a25c5c6345ffb7fa'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:49.442818 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-865b3912-f00e-4b04-a5f5-d458b64a7392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a915b0ed19e84f04a25c5c6345ffb7fa'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:49.443464 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-865b3912-f00e-4b04-a5f5-d458b64a7392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a915b0ed19e84f04a25c5c6345ffb7fa'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:49.458746 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1394/2817] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:59:49 2022] DELETE /identity/v3/users/a915b0ed19e84f04a25c5c6345ffb7fa => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:49.463295 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd519c11-9bac-4f5a-9d1c-0cf3c077aeef None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:49.463849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd519c11-9bac-4f5a-9d1c-0cf3c077aeef None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:49.463849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd519c11-9bac-4f5a-9d1c-0cf3c077aeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:49.486721 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-bd519c11-9bac-4f5a-9d1c-0cf3c077aeef None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:49.488156 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-bd519c11-9bac-4f5a-9d1c-0cf3c077aeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:49.563728 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1424/2818] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:59:49.573858 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b3912-f00e-4b04-a5f5-d458b64a7392 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:49.575337 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-865b3912-f00e-4b04-a5f5-d458b64a7392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:49.663953 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b3912-f00e-4b04-a5f5-d458b64a7392 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:49.668368 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871bd30-ec16-4207-b3be-720c5d8751d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:49.669045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871bd30-ec16-4207-b3be-720c5d8751d3 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:49.669388 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871bd30-ec16-4207-b3be-720c5d8751d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:49.669674 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871bd30-ec16-4207-b3be-720c5d8751d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:49.680989 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871bd30-ec16-4207-b3be-720c5d8751d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:49.682668 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871bd30-ec16-4207-b3be-720c5d8751d3 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:49.695180 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1395/2819] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:59:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:59:50.024519 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd519c11-9bac-4f5a-9d1c-0cf3c077aeef None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:50.027011 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-bd519c11-9bac-4f5a-9d1c-0cf3c077aeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:50.046751 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd519c11-9bac-4f5a-9d1c-0cf3c077aeef None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:50.054060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e476a0-f75b-4c61-91af-36a4fc4768e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:50.055081 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e476a0-f75b-4c61-91af-36a4fc4768e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:50.055486 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e476a0-f75b-4c61-91af-36a4fc4768e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:50.055880 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e476a0-f75b-4c61-91af-36a4fc4768e3 admin admin] PATH_INFO: `/v3/projects/bf1548c988f94ff9b4bd15a848825365` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:50.057407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e476a0-f75b-4c61-91af-36a4fc4768e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf1548c988f94ff9b4bd15a848825365)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:50.060377 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e476a0-f75b-4c61-91af-36a4fc4768e3 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:50.086302 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:50.127719 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-79e476a0-f75b-4c61-91af-36a4fc4768e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf1548c988f94ff9b4bd15a848825365', 'request_id': 'req-79e476a0-f75b-4c61-91af-36a4fc4768e3'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:50.135027 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-79e476a0-f75b-4c61-91af-36a4fc4768e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf1548c988f94ff9b4bd15a848825365', 'request_id': 'req-79e476a0-f75b-4c61-91af-36a4fc4768e3'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:50.145613 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-79e476a0-f75b-4c61-91af-36a4fc4768e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf1548c988f94ff9b4bd15a848825365', 'request_id': 'req-79e476a0-f75b-4c61-91af-36a4fc4768e3'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:50.158846 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-79e476a0-f75b-4c61-91af-36a4fc4768e3 admin admin] The token cache is being invalidate because project bf1548c988f94ff9b4bd15a848825365 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:59:50.159145 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-79e476a0-f75b-4c61-91af-36a4fc4768e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:50.159842 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-79e476a0-f75b-4c61-91af-36a4fc4768e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:50.161471 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1425/2820] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:59:50 2022] DELETE /identity/v3/projects/bf1548c988f94ff9b4bd15a848825365 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:50.166109 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871bd30-ec16-4207-b3be-720c5d8751d3 service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:50.179498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871bd30-ec16-4207-b3be-720c5d8751d3 service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:50.255769 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b15b14-30d5-4555-9ada-6eac02025937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:50.256855 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b15b14-30d5-4555-9ada-6eac02025937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:50.257201 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b15b14-30d5-4555-9ada-6eac02025937 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:50.257586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b15b14-30d5-4555-9ada-6eac02025937 admin admin] PATH_INFO: `/v3/users/c3de986750794d12934319eea0557b44` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:50.259254 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b15b14-30d5-4555-9ada-6eac02025937 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3de986750794d12934319eea0557b44)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:50.261726 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b15b14-30d5-4555-9ada-6eac02025937 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:50.311805 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:50.338979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-78b15b14-30d5-4555-9ada-6eac02025937 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3de986750794d12934319eea0557b44'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:50.356038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-78b15b14-30d5-4555-9ada-6eac02025937 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3de986750794d12934319eea0557b44'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:50.364932 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-78b15b14-30d5-4555-9ada-6eac02025937 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3de986750794d12934319eea0557b44'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:50.365810 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-78b15b14-30d5-4555-9ada-6eac02025937 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3de986750794d12934319eea0557b44'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:50.366474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-78b15b14-30d5-4555-9ada-6eac02025937 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3de986750794d12934319eea0557b44'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:50.381926 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1396/2821] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:59:50 2022] DELETE /identity/v3/users/c3de986750794d12934319eea0557b44 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:50.694563 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e476a0-f75b-4c61-91af-36a4fc4768e3 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:50.696349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-79e476a0-f75b-4c61-91af-36a4fc4768e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:50.715637 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e476a0-f75b-4c61-91af-36a4fc4768e3 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:50.806181 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df09ffd-e1f0-479c-bc7e-2ed0daa8e4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:50.807193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df09ffd-e1f0-479c-bc7e-2ed0daa8e4a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:50.807534 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df09ffd-e1f0-479c-bc7e-2ed0daa8e4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:50.807830 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df09ffd-e1f0-479c-bc7e-2ed0daa8e4a1 admin admin] PATH_INFO: `/v3/projects/7eb5e4290a0d40c6aee41776c3a34f48` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:50.809282 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df09ffd-e1f0-479c-bc7e-2ed0daa8e4a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eb5e4290a0d40c6aee41776c3a34f48)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:50.811336 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df09ffd-e1f0-479c-bc7e-2ed0daa8e4a1 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:50.873461 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4df09ffd-e1f0-479c-bc7e-2ed0daa8e4a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eb5e4290a0d40c6aee41776c3a34f48', 'request_id': 'req-4df09ffd-e1f0-479c-bc7e-2ed0daa8e4a1'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:50.885037 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4df09ffd-e1f0-479c-bc7e-2ed0daa8e4a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eb5e4290a0d40c6aee41776c3a34f48', 'request_id': 'req-4df09ffd-e1f0-479c-bc7e-2ed0daa8e4a1'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:50.896556 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4df09ffd-e1f0-479c-bc7e-2ed0daa8e4a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eb5e4290a0d40c6aee41776c3a34f48', 'request_id': 'req-4df09ffd-e1f0-479c-bc7e-2ed0daa8e4a1'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:50.910298 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4df09ffd-e1f0-479c-bc7e-2ed0daa8e4a1 admin admin] The token cache is being invalidate because project 7eb5e4290a0d40c6aee41776c3a34f48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:59:50.910616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4df09ffd-e1f0-479c-bc7e-2ed0daa8e4a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:50.911808 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-4df09ffd-e1f0-479c-bc7e-2ed0daa8e4a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:50.913996 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1426/2822] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:59:50 2022] DELETE /identity/v3/projects/7eb5e4290a0d40c6aee41776c3a34f48 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:50.918474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b15b14-30d5-4555-9ada-6eac02025937 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:50.919945 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-78b15b14-30d5-4555-9ada-6eac02025937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:50.934710 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b15b14-30d5-4555-9ada-6eac02025937 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:51.008146 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0227d-c1f6-4a16-94f1-cc8f7fe847d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:51.009935 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0227d-c1f6-4a16-94f1-cc8f7fe847d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:51.010381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0227d-c1f6-4a16-94f1-cc8f7fe847d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:51.010886 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0227d-c1f6-4a16-94f1-cc8f7fe847d0 admin admin] PATH_INFO: `/v3/users/ea89ecf7a3164958918ff93e34e3d477` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:51.015403 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0227d-c1f6-4a16-94f1-cc8f7fe847d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea89ecf7a3164958918ff93e34e3d477)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:51.018115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0227d-c1f6-4a16-94f1-cc8f7fe847d0 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:51.077815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-8ee0227d-c1f6-4a16-94f1-cc8f7fe847d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea89ecf7a3164958918ff93e34e3d477'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:51.099153 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-8ee0227d-c1f6-4a16-94f1-cc8f7fe847d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea89ecf7a3164958918ff93e34e3d477'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:51.111453 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-8ee0227d-c1f6-4a16-94f1-cc8f7fe847d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea89ecf7a3164958918ff93e34e3d477'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:51.112425 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-8ee0227d-c1f6-4a16-94f1-cc8f7fe847d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea89ecf7a3164958918ff93e34e3d477'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:51.113557 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-8ee0227d-c1f6-4a16-94f1-cc8f7fe847d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea89ecf7a3164958918ff93e34e3d477'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:51.129852 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1397/2823] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 15:59:50 2022] DELETE /identity/v3/users/ea89ecf7a3164958918ff93e34e3d477 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:51.546591 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df09ffd-e1f0-479c-bc7e-2ed0daa8e4a1 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:51.549534 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-4df09ffd-e1f0-479c-bc7e-2ed0daa8e4a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:51.572403 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df09ffd-e1f0-479c-bc7e-2ed0daa8e4a1 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:51.615781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:51.648746 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86baec1-0fe2-470e-9881-108664fa1866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:51.649653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86baec1-0fe2-470e-9881-108664fa1866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:51.649948 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86baec1-0fe2-470e-9881-108664fa1866 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:51.650231 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86baec1-0fe2-470e-9881-108664fa1866 admin admin] PATH_INFO: `/v3/projects/fab888eaf583424e87326f2af137a9d3` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:51.651489 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86baec1-0fe2-470e-9881-108664fa1866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fab888eaf583424e87326f2af137a9d3)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:51.653492 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86baec1-0fe2-470e-9881-108664fa1866 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:51.710691 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-a86baec1-0fe2-470e-9881-108664fa1866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fab888eaf583424e87326f2af137a9d3', 'request_id': 'req-a86baec1-0fe2-470e-9881-108664fa1866'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:51.718612 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-a86baec1-0fe2-470e-9881-108664fa1866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fab888eaf583424e87326f2af137a9d3', 'request_id': 'req-a86baec1-0fe2-470e-9881-108664fa1866'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:51.727912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-a86baec1-0fe2-470e-9881-108664fa1866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fab888eaf583424e87326f2af137a9d3', 'request_id': 'req-a86baec1-0fe2-470e-9881-108664fa1866'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:51.740315 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-a86baec1-0fe2-470e-9881-108664fa1866 admin admin] The token cache is being invalidate because project fab888eaf583424e87326f2af137a9d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91942) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 15:59:51.740608 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-a86baec1-0fe2-470e-9881-108664fa1866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:51.741304 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-a86baec1-0fe2-470e-9881-108664fa1866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 15:59:51.743053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1427/2824] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 15:59:51 2022] DELETE /identity/v3/projects/fab888eaf583424e87326f2af137a9d3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:51.748780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3f3f2-eb03-4a86-b994-a9dca963de2b None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:51.749367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3f3f2-eb03-4a86-b994-a9dca963de2b None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:51.749367 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3f3f2-eb03-4a86-b994-a9dca963de2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:51.772186 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-75e3f3f2-eb03-4a86-b994-a9dca963de2b None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:51.773450 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-75e3f3f2-eb03-4a86-b994-a9dca963de2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:51.797298 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:51.861171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1398/2825] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 15:59:51.871876 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4823a6-d698-456c-8dfb-ddc61dd8c6d7 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:51.872190 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4823a6-d698-456c-8dfb-ddc61dd8c6d7 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:51.872484 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4823a6-d698-456c-8dfb-ddc61dd8c6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:51.896430 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-8d4823a6-d698-456c-8dfb-ddc61dd8c6d7 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:51.897783 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8d4823a6-d698-456c-8dfb-ddc61dd8c6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:51.907619 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1428/2826] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:59:51.912433 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3f3f2-eb03-4a86-b994-a9dca963de2b None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:51.924639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3f3f2-eb03-4a86-b994-a9dca963de2b None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:51.928737 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6626272-4383-426e-9955-a9cc12b1f9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:51.929416 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6626272-4383-426e-9955-a9cc12b1f9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:51.929822 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6626272-4383-426e-9955-a9cc12b1f9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:51.930088 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6626272-4383-426e-9955-a9cc12b1f9d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:51.930828 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6626272-4383-426e-9955-a9cc12b1f9d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91941) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 15:59:51.931133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6626272-4383-426e-9955-a9cc12b1f9d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:51.932755 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6626272-4383-426e-9955-a9cc12b1f9d3 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:51.946314 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1399/2827] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 15:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 15:59:51.951617 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4823a6-d698-456c-8dfb-ddc61dd8c6d7 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:51.970771 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4823a6-d698-456c-8dfb-ddc61dd8c6d7 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:51.978948 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d8096-413a-4e3c-9f06-84fc05c976d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:51.980046 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d8096-413a-4e3c-9f06-84fc05c976d0 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:51.980459 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d8096-413a-4e3c-9f06-84fc05c976d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:51.980752 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d8096-413a-4e3c-9f06-84fc05c976d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:51.982078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d8096-413a-4e3c-9f06-84fc05c976d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:51.986407 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d8096-413a-4e3c-9f06-84fc05c976d0 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:52.023095 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1429/2828] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:59:51 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:59:52.028079 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6626272-4383-426e-9955-a9cc12b1f9d3 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:52.040605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6626272-4383-426e-9955-a9cc12b1f9d3 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:52.113082 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85866715-f07e-49d0-ac6a-8cc3d7fb30a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:52.113860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85866715-f07e-49d0-ac6a-8cc3d7fb30a2 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:52.114148 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85866715-f07e-49d0-ac6a-8cc3d7fb30a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:52.114426 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85866715-f07e-49d0-ac6a-8cc3d7fb30a2 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:52.115282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85866715-f07e-49d0-ac6a-8cc3d7fb30a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:52.117564 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85866715-f07e-49d0-ac6a-8cc3d7fb30a2 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:52.145130 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1400/2829] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:59:52 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:59:52.149743 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d8096-413a-4e3c-9f06-84fc05c976d0 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:52.162503 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d8096-413a-4e3c-9f06-84fc05c976d0 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:52.167768 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17639458-254a-48c6-a525-f93e8307e36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:52.168463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17639458-254a-48c6-a525-f93e8307e36f admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:52.168768 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17639458-254a-48c6-a525-f93e8307e36f admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:52.169146 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17639458-254a-48c6-a525-f93e8307e36f admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:52.169932 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17639458-254a-48c6-a525-f93e8307e36f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:52.171921 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17639458-254a-48c6-a525-f93e8307e36f admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:52.184718 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1430/2830] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:59:52.191198 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85866715-f07e-49d0-ac6a-8cc3d7fb30a2 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:52.208769 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85866715-f07e-49d0-ac6a-8cc3d7fb30a2 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:52.215350 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba9259-5e05-4ba2-9aaf-922d6b5bd9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:52.216079 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba9259-5e05-4ba2-9aaf-922d6b5bd9c6 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:52.216369 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba9259-5e05-4ba2-9aaf-922d6b5bd9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:52.216799 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba9259-5e05-4ba2-9aaf-922d6b5bd9c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:52.217783 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba9259-5e05-4ba2-9aaf-922d6b5bd9c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:52.221276 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba9259-5e05-4ba2-9aaf-922d6b5bd9c6 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:52.236326 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1401/2831] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:59:52.240952 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17639458-254a-48c6-a525-f93e8307e36f admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:52.254879 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17639458-254a-48c6-a525-f93e8307e36f admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:52.260156 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d417e3-06b8-4880-968d-c7ed7e0bb795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:52.261120 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d417e3-06b8-4880-968d-c7ed7e0bb795 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:52.261500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d417e3-06b8-4880-968d-c7ed7e0bb795 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:52.261823 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d417e3-06b8-4880-968d-c7ed7e0bb795 admin admin] PATH_INFO: `/v3/projects/8d60d25dff9a4b32b1e5cd12c4b1a3ea/users/d27a217c30284e7eb6a340aba09b3d61/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:52.282053 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d417e3-06b8-4880-968d-c7ed7e0bb795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d60d25dff9a4b32b1e5cd12c4b1a3ea, user_id=d27a217c30284e7eb6a340aba09b3d61, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:52.284852 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d417e3-06b8-4880-968d-c7ed7e0bb795 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:52.300253 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1431/2832] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:59:52 2022] PUT /identity/v3/projects/8d60d25dff9a4b32b1e5cd12c4b1a3ea/users/d27a217c30284e7eb6a340aba09b3d61/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:52.365705 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9ca25-b8a1-454d-80d4-0639bd376107 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:52.366115 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9ca25-b8a1-454d-80d4-0639bd376107 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:52.366483 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9ca25-b8a1-454d-80d4-0639bd376107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:52.396536 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-2df9ca25-b8a1-454d-80d4-0639bd376107 None None] MFA Rules not processed for user `d27a217c30284e7eb6a340aba09b3d61`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:52.397813 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-2df9ca25-b8a1-454d-80d4-0639bd376107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:52.467569 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1402/2833] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:59:52.473375 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d417e3-06b8-4880-968d-c7ed7e0bb795 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:52.554743 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d417e3-06b8-4880-968d-c7ed7e0bb795 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:52.559962 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3196c1-0149-4455-83d7-eb93a3d288ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:52.560640 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3196c1-0149-4455-83d7-eb93a3d288ec admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:52.560922 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3196c1-0149-4455-83d7-eb93a3d288ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:52.561198 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3196c1-0149-4455-83d7-eb93a3d288ec admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:52.561940 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3196c1-0149-4455-83d7-eb93a3d288ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:52.564406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3196c1-0149-4455-83d7-eb93a3d288ec admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:52.568188 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:52.596287 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1432/2834] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:59:52 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:59:52.600518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9ca25-b8a1-454d-80d4-0639bd376107 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:52.612484 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9ca25-b8a1-454d-80d4-0639bd376107 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:52.638312 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:52.690080 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d406a-5ab1-43a4-b80b-6b203d614e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:52.691058 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d406a-5ab1-43a4-b80b-6b203d614e1c admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:52.691523 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d406a-5ab1-43a4-b80b-6b203d614e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:52.691815 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d406a-5ab1-43a4-b80b-6b203d614e1c admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:52.692739 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d406a-5ab1-43a4-b80b-6b203d614e1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:52.695620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d406a-5ab1-43a4-b80b-6b203d614e1c admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:52.721561 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1403/2835] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:59:52 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:59:52.726060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3196c1-0149-4455-83d7-eb93a3d288ec admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:52.737150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3196c1-0149-4455-83d7-eb93a3d288ec admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:52.742021 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b712057-71c9-43a7-b4f4-2c0e407016b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:52.742707 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b712057-71c9-43a7-b4f4-2c0e407016b2 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:52.742995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b712057-71c9-43a7-b4f4-2c0e407016b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:52.743265 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b712057-71c9-43a7-b4f4-2c0e407016b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:52.744001 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b712057-71c9-43a7-b4f4-2c0e407016b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:52.745630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b712057-71c9-43a7-b4f4-2c0e407016b2 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:52.757876 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1433/2836] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:59:52.762520 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d406a-5ab1-43a4-b80b-6b203d614e1c admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:52.773915 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d406a-5ab1-43a4-b80b-6b203d614e1c admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:52.778785 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f4f2a-d970-431b-bbae-a5c353c93e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:52.779478 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f4f2a-d970-431b-bbae-a5c353c93e4d admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:52.779772 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f4f2a-d970-431b-bbae-a5c353c93e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:52.780056 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f4f2a-d970-431b-bbae-a5c353c93e4d admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:52.780797 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f4f2a-d970-431b-bbae-a5c353c93e4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:52.782513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f4f2a-d970-431b-bbae-a5c353c93e4d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:52.793568 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1404/2837] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:59:52.798414 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b712057-71c9-43a7-b4f4-2c0e407016b2 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:52.808631 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b712057-71c9-43a7-b4f4-2c0e407016b2 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:52.813924 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb39a0-dc4a-4b8c-9d76-b2f83f7bcff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:52.814819 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb39a0-dc4a-4b8c-9d76-b2f83f7bcff5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:52.815103 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb39a0-dc4a-4b8c-9d76-b2f83f7bcff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:52.815381 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb39a0-dc4a-4b8c-9d76-b2f83f7bcff5 admin admin] PATH_INFO: `/v3/projects/25d3037ddf254c7397dfb15523af02be/users/cceac7950df4486ba1f11f7287cc3e1e/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:52.832590 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb39a0-dc4a-4b8c-9d76-b2f83f7bcff5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25d3037ddf254c7397dfb15523af02be, user_id=cceac7950df4486ba1f11f7287cc3e1e, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:52.835328 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb39a0-dc4a-4b8c-9d76-b2f83f7bcff5 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:52.849691 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1434/2838] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:59:52 2022] PUT /identity/v3/projects/25d3037ddf254c7397dfb15523af02be/users/cceac7950df4486ba1f11f7287cc3e1e/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:52.864174 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff47d72-6b99-4d5d-ba98-0d88e8713e9b None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:52.864655 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff47d72-6b99-4d5d-ba98-0d88e8713e9b None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:52.865085 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff47d72-6b99-4d5d-ba98-0d88e8713e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:52.894322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-4ff47d72-6b99-4d5d-ba98-0d88e8713e9b None None] MFA Rules not processed for user `cceac7950df4486ba1f11f7287cc3e1e`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:52.895577 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4ff47d72-6b99-4d5d-ba98-0d88e8713e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:52.969935 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1405/2839] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:59:52.974561 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb39a0-dc4a-4b8c-9d76-b2f83f7bcff5 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:53.043966 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb39a0-dc4a-4b8c-9d76-b2f83f7bcff5 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:53.049671 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69633c-dddf-4914-88f3-34798aa4043d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:53.050371 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69633c-dddf-4914-88f3-34798aa4043d admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:53.050661 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69633c-dddf-4914-88f3-34798aa4043d admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:53.050946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69633c-dddf-4914-88f3-34798aa4043d admin admin] PATH_INFO: `/v3/projects` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:53.051653 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69633c-dddf-4914-88f3-34798aa4043d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:53.053616 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69633c-dddf-4914-88f3-34798aa4043d admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:53.083398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1435/2840] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 15:59:52 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:59:53.088256 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff47d72-6b99-4d5d-ba98-0d88e8713e9b None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:53.100737 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff47d72-6b99-4d5d-ba98-0d88e8713e9b None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:53.162013 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ba420-d95c-4307-8067-3d4efd781d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:53.162701 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ba420-d95c-4307-8067-3d4efd781d32 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:53.162983 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ba420-d95c-4307-8067-3d4efd781d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:53.163258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ba420-d95c-4307-8067-3d4efd781d32 admin admin] PATH_INFO: `/v3/users` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:53.163965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ba420-d95c-4307-8067-3d4efd781d32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:53.165920 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ba420-d95c-4307-8067-3d4efd781d32 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:53.187628 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1406/2841] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 15:59:53 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 15:59:53.191801 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69633c-dddf-4914-88f3-34798aa4043d admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:53.202810 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69633c-dddf-4914-88f3-34798aa4043d admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:53.208409 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6ce65-e730-48d5-8374-7f0f0ca56e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:53.209097 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6ce65-e730-48d5-8374-7f0f0ca56e73 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:53.209386 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6ce65-e730-48d5-8374-7f0f0ca56e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:53.209776 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6ce65-e730-48d5-8374-7f0f0ca56e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:53.210526 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6ce65-e730-48d5-8374-7f0f0ca56e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:53.212147 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6ce65-e730-48d5-8374-7f0f0ca56e73 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:53.224827 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1436/2842] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:59:53.229111 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ba420-d95c-4307-8067-3d4efd781d32 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:53.240189 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ba420-d95c-4307-8067-3d4efd781d32 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:53.245353 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811eeeb-7d29-4027-a678-9f9b3040a12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:53.246297 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811eeeb-7d29-4027-a678-9f9b3040a12d admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:53.246586 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811eeeb-7d29-4027-a678-9f9b3040a12d admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:53.246867 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811eeeb-7d29-4027-a678-9f9b3040a12d admin admin] PATH_INFO: `/v3/projects/1b5fff9742904dd1afd00272558fbef6/users/22e31f0698564f4cb5f72cd1e6adf312/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:53.263313 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811eeeb-7d29-4027-a678-9f9b3040a12d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b5fff9742904dd1afd00272558fbef6, user_id=22e31f0698564f4cb5f72cd1e6adf312, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:53.267009 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811eeeb-7d29-4027-a678-9f9b3040a12d admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:53.269605 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:53.285143 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1407/2843] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:59:53 2022] PUT /identity/v3/projects/1b5fff9742904dd1afd00272558fbef6/users/22e31f0698564f4cb5f72cd1e6adf312/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:53.289537 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6ce65-e730-48d5-8374-7f0f0ca56e73 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:53.302392 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6ce65-e730-48d5-8374-7f0f0ca56e73 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:53.364821 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d01e7c-1b7a-498e-8d24-fb0221174ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:53.365508 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d01e7c-1b7a-498e-8d24-fb0221174ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:53.365838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d01e7c-1b7a-498e-8d24-fb0221174ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:53.366124 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d01e7c-1b7a-498e-8d24-fb0221174ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:53.366857 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d01e7c-1b7a-498e-8d24-fb0221174ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:53.368478 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d01e7c-1b7a-498e-8d24-fb0221174ab9 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:53.379993 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1437/2844] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 15:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 15:59:53.384677 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811eeeb-7d29-4027-a678-9f9b3040a12d admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:53.396365 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811eeeb-7d29-4027-a678-9f9b3040a12d admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:53.401225 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f676d86-cef6-4c58-be91-60e3c5359e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:53.402179 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f676d86-cef6-4c58-be91-60e3c5359e1b admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:53.402498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f676d86-cef6-4c58-be91-60e3c5359e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:53.402777 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f676d86-cef6-4c58-be91-60e3c5359e1b admin admin] PATH_INFO: `/v3/projects/1b5fff9742904dd1afd00272558fbef6/users/22e31f0698564f4cb5f72cd1e6adf312/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:53.404876 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f676d86-cef6-4c58-be91-60e3c5359e1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b5fff9742904dd1afd00272558fbef6, user_id=22e31f0698564f4cb5f72cd1e6adf312, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:53.407592 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f676d86-cef6-4c58-be91-60e3c5359e1b admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:53.418870 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1408/2845] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 15:59:53 2022] PUT /identity/v3/projects/1b5fff9742904dd1afd00272558fbef6/users/22e31f0698564f4cb5f72cd1e6adf312/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 15:59:53.429912 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497cfab-0e1f-4ba5-be4e-e81dde522fdc None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:53.430213 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497cfab-0e1f-4ba5-be4e-e81dde522fdc None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:53.430495 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497cfab-0e1f-4ba5-be4e-e81dde522fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:53.461107 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-2497cfab-0e1f-4ba5-be4e-e81dde522fdc None None] MFA Rules not processed for user `22e31f0698564f4cb5f72cd1e6adf312`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 15:59:53.462379 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2497cfab-0e1f-4ba5-be4e-e81dde522fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:53.471710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 15:59:53.538206 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1438/2846] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 15:59:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 15:59:53.549778 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f676d86-cef6-4c58-be91-60e3c5359e1b admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:53.551572 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4f676d86-cef6-4c58-be91-60e3c5359e1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:53.617899 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f676d86-cef6-4c58-be91-60e3c5359e1b admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:53.621691 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9566a-4aee-47ea-bffd-ad5e3d67daaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:53.622362 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9566a-4aee-47ea-bffd-ad5e3d67daaa service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:53.622650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9566a-4aee-47ea-bffd-ad5e3d67daaa service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:53.622934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9566a-4aee-47ea-bffd-ad5e3d67daaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:53.685094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9566a-4aee-47ea-bffd-ad5e3d67daaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:53.686606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9566a-4aee-47ea-bffd-ad5e3d67daaa service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:53.697716 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1409/2847] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:59:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:59:55.255286 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497cfab-0e1f-4ba5-be4e-e81dde522fdc None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:55.256972 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2497cfab-0e1f-4ba5-be4e-e81dde522fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 15:59:55.278927 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497cfab-0e1f-4ba5-be4e-e81dde522fdc None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:55.283582 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9c944-27b5-45ef-a3f7-f0e3545ffd34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:55.284269 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9c944-27b5-45ef-a3f7-f0e3545ffd34 service cinder] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:55.284560 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9c944-27b5-45ef-a3f7-f0e3545ffd34 service cinder] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:55.284844 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9c944-27b5-45ef-a3f7-f0e3545ffd34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:55.350881 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9c944-27b5-45ef-a3f7-f0e3545ffd34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:55.352248 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9c944-27b5-45ef-a3f7-f0e3545ffd34 service cinder] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:55.362354 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1439/2848] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:59:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 15:59:55.949603 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9566a-4aee-47ea-bffd-ad5e3d67daaa service cinder] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 15:59:55.964591 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9566a-4aee-47ea-bffd-ad5e3d67daaa service cinder] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 15:59:55.971780 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7354d17-61ff-4583-8362-d209a23b9b80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 15:59:55.972902 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7354d17-61ff-4583-8362-d209a23b9b80 service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 15:59:55.973397 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7354d17-61ff-4583-8362-d209a23b9b80 service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 15:59:55.973951 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7354d17-61ff-4583-8362-d209a23b9b80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 15:59:55.991254 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7354d17-61ff-4583-8362-d209a23b9b80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 15:59:55.993322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7354d17-61ff-4583-8362-d209a23b9b80 service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 15:59:56.010689 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1410/2849] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 15:59:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 16:00:00.134148 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9c944-27b5-45ef-a3f7-f0e3545ffd34 service cinder] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:00.208849 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9c944-27b5-45ef-a3f7-f0e3545ffd34 service cinder] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:00.213599 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae9858-b2f7-4119-a589-677a57929147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:00.214521 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae9858-b2f7-4119-a589-677a57929147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:00.214809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae9858-b2f7-4119-a589-677a57929147 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:00.215091 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae9858-b2f7-4119-a589-677a57929147 admin admin] PATH_INFO: `/v3/users/d27a217c30284e7eb6a340aba09b3d61` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:00.216331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae9858-b2f7-4119-a589-677a57929147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d27a217c30284e7eb6a340aba09b3d61)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:00.218968 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae9858-b2f7-4119-a589-677a57929147 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:00.274045 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:00:00.275970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5aae9858-b2f7-4119-a589-677a57929147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd27a217c30284e7eb6a340aba09b3d61'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:00.291001 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5aae9858-b2f7-4119-a589-677a57929147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd27a217c30284e7eb6a340aba09b3d61'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:00.291757 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5aae9858-b2f7-4119-a589-677a57929147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd27a217c30284e7eb6a340aba09b3d61'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:00.302099 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5aae9858-b2f7-4119-a589-677a57929147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd27a217c30284e7eb6a340aba09b3d61'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:00.315791 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-5aae9858-b2f7-4119-a589-677a57929147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd27a217c30284e7eb6a340aba09b3d61'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:00.327066 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1440/2850] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 16:00:00 2022] DELETE /identity/v3/users/d27a217c30284e7eb6a340aba09b3d61 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 16:00:00.333983 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f80eb6-2a64-4e8f-992e-e7d26ef803cf None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:00.334783 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f80eb6-2a64-4e8f-992e-e7d26ef803cf None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:00.334783 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f80eb6-2a64-4e8f-992e-e7d26ef803cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:00.354099 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-d3f80eb6-2a64-4e8f-992e-e7d26ef803cf None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 16:00:00.355452 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d3f80eb6-2a64-4e8f-992e-e7d26ef803cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:00.427094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1411/2851] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 16:00:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 16:00:00.437481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae9858-b2f7-4119-a589-677a57929147 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:00.439753 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-5aae9858-b2f7-4119-a589-677a57929147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:00.507114 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae9858-b2f7-4119-a589-677a57929147 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:00.511196 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d9201-3599-4ca1-938c-31d9f872d5e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:00.511866 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d9201-3599-4ca1-938c-31d9f872d5e7 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:00.512152 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d9201-3599-4ca1-938c-31d9f872d5e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:00.512432 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d9201-3599-4ca1-938c-31d9f872d5e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:00.522828 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d9201-3599-4ca1-938c-31d9f872d5e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:00.524302 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d9201-3599-4ca1-938c-31d9f872d5e7 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:00.535720 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1441/2852] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 16:00:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 16:00:00.879043 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f80eb6-2a64-4e8f-992e-e7d26ef803cf None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:00.881537 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d3f80eb6-2a64-4e8f-992e-e7d26ef803cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:00.902349 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f80eb6-2a64-4e8f-992e-e7d26ef803cf None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:00.910864 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5a86c-4c5d-4310-bf99-a29fb8f954d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:00.912358 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5a86c-4c5d-4310-bf99-a29fb8f954d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:00.912875 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5a86c-4c5d-4310-bf99-a29fb8f954d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:00.913329 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5a86c-4c5d-4310-bf99-a29fb8f954d9 admin admin] PATH_INFO: `/v3/projects/8d60d25dff9a4b32b1e5cd12c4b1a3ea` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:00.915610 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5a86c-4c5d-4310-bf99-a29fb8f954d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d60d25dff9a4b32b1e5cd12c4b1a3ea)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:00.919366 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5a86c-4c5d-4310-bf99-a29fb8f954d9 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:00.943005 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:00:00.988625 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a4f5a86c-4c5d-4310-bf99-a29fb8f954d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d60d25dff9a4b32b1e5cd12c4b1a3ea', 'request_id': 'req-a4f5a86c-4c5d-4310-bf99-a29fb8f954d9'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:00.995681 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a4f5a86c-4c5d-4310-bf99-a29fb8f954d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d60d25dff9a4b32b1e5cd12c4b1a3ea', 'request_id': 'req-a4f5a86c-4c5d-4310-bf99-a29fb8f954d9'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:01.002486 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a4f5a86c-4c5d-4310-bf99-a29fb8f954d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d60d25dff9a4b32b1e5cd12c4b1a3ea', 'request_id': 'req-a4f5a86c-4c5d-4310-bf99-a29fb8f954d9'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:01.019681 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a4f5a86c-4c5d-4310-bf99-a29fb8f954d9 admin admin] The token cache is being invalidate because project 8d60d25dff9a4b32b1e5cd12c4b1a3ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 16:00:01.020159 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a4f5a86c-4c5d-4310-bf99-a29fb8f954d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:01.021295 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-a4f5a86c-4c5d-4310-bf99-a29fb8f954d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:01.023330 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1412/2853] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 16:00:00 2022] DELETE /identity/v3/projects/8d60d25dff9a4b32b1e5cd12c4b1a3ea => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 16:00:01.027923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d9201-3599-4ca1-938c-31d9f872d5e7 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:01.038363 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d9201-3599-4ca1-938c-31d9f872d5e7 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:01.107810 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f7c61-2b6d-441c-96df-51b8b2808d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:01.108895 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f7c61-2b6d-441c-96df-51b8b2808d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:01.109288 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f7c61-2b6d-441c-96df-51b8b2808d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:01.109676 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f7c61-2b6d-441c-96df-51b8b2808d2a admin admin] PATH_INFO: `/v3/users/cceac7950df4486ba1f11f7287cc3e1e` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:01.111245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f7c61-2b6d-441c-96df-51b8b2808d2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cceac7950df4486ba1f11f7287cc3e1e)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:01.114236 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f7c61-2b6d-441c-96df-51b8b2808d2a admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:01.194535 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-783f7c61-2b6d-441c-96df-51b8b2808d2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cceac7950df4486ba1f11f7287cc3e1e'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:01.222709 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-783f7c61-2b6d-441c-96df-51b8b2808d2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cceac7950df4486ba1f11f7287cc3e1e'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:01.223890 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-783f7c61-2b6d-441c-96df-51b8b2808d2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cceac7950df4486ba1f11f7287cc3e1e'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:01.229461 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:00:01.237866 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-783f7c61-2b6d-441c-96df-51b8b2808d2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cceac7950df4486ba1f11f7287cc3e1e'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:01.252400 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-783f7c61-2b6d-441c-96df-51b8b2808d2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cceac7950df4486ba1f11f7287cc3e1e'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:01.261955 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1442/2854] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 16:00:01 2022] DELETE /identity/v3/users/cceac7950df4486ba1f11f7287cc3e1e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 16:00:01.674660 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5a86c-4c5d-4310-bf99-a29fb8f954d9 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:01.677257 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a4f5a86c-4c5d-4310-bf99-a29fb8f954d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:01.695418 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5a86c-4c5d-4310-bf99-a29fb8f954d9 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:01.775921 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b3481-4785-4e01-bdbc-84320da81dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:01.776814 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b3481-4785-4e01-bdbc-84320da81dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:01.777099 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b3481-4785-4e01-bdbc-84320da81dee admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:01.777381 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b3481-4785-4e01-bdbc-84320da81dee admin admin] PATH_INFO: `/v3/projects/25d3037ddf254c7397dfb15523af02be` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:01.778713 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b3481-4785-4e01-bdbc-84320da81dee admin admin] RBAC: Authorizing `identity:delete_project(project_id=25d3037ddf254c7397dfb15523af02be)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:01.780713 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b3481-4785-4e01-bdbc-84320da81dee admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:01.844094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d42b3481-4785-4e01-bdbc-84320da81dee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25d3037ddf254c7397dfb15523af02be', 'request_id': 'req-d42b3481-4785-4e01-bdbc-84320da81dee'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:01.851934 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d42b3481-4785-4e01-bdbc-84320da81dee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25d3037ddf254c7397dfb15523af02be', 'request_id': 'req-d42b3481-4785-4e01-bdbc-84320da81dee'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:01.860183 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d42b3481-4785-4e01-bdbc-84320da81dee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25d3037ddf254c7397dfb15523af02be', 'request_id': 'req-d42b3481-4785-4e01-bdbc-84320da81dee'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:01.875955 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d42b3481-4785-4e01-bdbc-84320da81dee admin admin] The token cache is being invalidate because project 25d3037ddf254c7397dfb15523af02be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 16:00:01.876252 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d42b3481-4785-4e01-bdbc-84320da81dee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:01.877071 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-d42b3481-4785-4e01-bdbc-84320da81dee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:01.878718 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1413/2855] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 16:00:01 2022] DELETE /identity/v3/projects/25d3037ddf254c7397dfb15523af02be => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 16:00:01.882939 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f7c61-2b6d-441c-96df-51b8b2808d2a admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:01.884481 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-783f7c61-2b6d-441c-96df-51b8b2808d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:01.898921 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f7c61-2b6d-441c-96df-51b8b2808d2a admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:01.970214 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59434840-6032-430c-9a11-9cf877bdcfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:01.971116 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59434840-6032-430c-9a11-9cf877bdcfdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:01.971406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59434840-6032-430c-9a11-9cf877bdcfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:01.971688 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59434840-6032-430c-9a11-9cf877bdcfdc admin admin] PATH_INFO: `/v3/users/22e31f0698564f4cb5f72cd1e6adf312` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:01.972956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59434840-6032-430c-9a11-9cf877bdcfdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=22e31f0698564f4cb5f72cd1e6adf312)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:01.975036 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59434840-6032-430c-9a11-9cf877bdcfdc admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:02.036515 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-59434840-6032-430c-9a11-9cf877bdcfdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22e31f0698564f4cb5f72cd1e6adf312'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:02.057109 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-59434840-6032-430c-9a11-9cf877bdcfdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22e31f0698564f4cb5f72cd1e6adf312'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:02.057915 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-59434840-6032-430c-9a11-9cf877bdcfdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22e31f0698564f4cb5f72cd1e6adf312'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:02.067158 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-59434840-6032-430c-9a11-9cf877bdcfdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22e31f0698564f4cb5f72cd1e6adf312'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:02.075941 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-59434840-6032-430c-9a11-9cf877bdcfdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22e31f0698564f4cb5f72cd1e6adf312'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:02.083690 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1443/2856] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 16:00:01 2022] DELETE /identity/v3/users/22e31f0698564f4cb5f72cd1e6adf312 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 16:00:02.422562 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b3481-4785-4e01-bdbc-84320da81dee admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:02.424459 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-d42b3481-4785-4e01-bdbc-84320da81dee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:02.439754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b3481-4785-4e01-bdbc-84320da81dee admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:02.483518 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:00:02.513881 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d78e63-9c35-4032-8fc0-f37f6c09177f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:02.514754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d78e63-9c35-4032-8fc0-f37f6c09177f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:02.515154 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d78e63-9c35-4032-8fc0-f37f6c09177f admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:02.515382 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d78e63-9c35-4032-8fc0-f37f6c09177f admin admin] PATH_INFO: `/v3/projects/1b5fff9742904dd1afd00272558fbef6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:02.517248 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d78e63-9c35-4032-8fc0-f37f6c09177f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b5fff9742904dd1afd00272558fbef6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:02.519434 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d78e63-9c35-4032-8fc0-f37f6c09177f admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:02.576466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e8d78e63-9c35-4032-8fc0-f37f6c09177f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b5fff9742904dd1afd00272558fbef6', 'request_id': 'req-e8d78e63-9c35-4032-8fc0-f37f6c09177f'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:02.584392 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e8d78e63-9c35-4032-8fc0-f37f6c09177f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b5fff9742904dd1afd00272558fbef6', 'request_id': 'req-e8d78e63-9c35-4032-8fc0-f37f6c09177f'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:02.590474 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e8d78e63-9c35-4032-8fc0-f37f6c09177f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b5fff9742904dd1afd00272558fbef6', 'request_id': 'req-e8d78e63-9c35-4032-8fc0-f37f6c09177f'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:02.606691 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e8d78e63-9c35-4032-8fc0-f37f6c09177f admin admin] The token cache is being invalidate because project 1b5fff9742904dd1afd00272558fbef6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 16:00:02.606965 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e8d78e63-9c35-4032-8fc0-f37f6c09177f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:02.607620 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-e8d78e63-9c35-4032-8fc0-f37f6c09177f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:02.609717 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1414/2857] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 16:00:02 2022] DELETE /identity/v3/projects/1b5fff9742904dd1afd00272558fbef6 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 16:00:02.615708 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80f1e6-34ca-4223-a96c-d3e4c399ad70 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:02.616557 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80f1e6-34ca-4223-a96c-d3e4c399ad70 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:02.616737 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80f1e6-34ca-4223-a96c-d3e4c399ad70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:02.636914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-8b80f1e6-34ca-4223-a96c-d3e4c399ad70 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 16:00:02.638239 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8b80f1e6-34ca-4223-a96c-d3e4c399ad70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:02.706658 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1444/2858] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 16:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 16:00:02.715003 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9247f-173e-4e0e-ad78-f67521aa6f25 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:02.715661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9247f-173e-4e0e-ad78-f67521aa6f25 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:02.715661 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9247f-173e-4e0e-ad78-f67521aa6f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:02.737740 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-5ac9247f-173e-4e0e-ad78-f67521aa6f25 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 16:00:02.739303 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-5ac9247f-173e-4e0e-ad78-f67521aa6f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:02.748038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1415/2859] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 16:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 16:00:02.753096 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80f1e6-34ca-4223-a96c-d3e4c399ad70 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:02.759822 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:00:02.764843 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80f1e6-34ca-4223-a96c-d3e4c399ad70 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:02.772916 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a02df-c0cb-4e10-b3d2-ea19de87ef8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:02.774065 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a02df-c0cb-4e10-b3d2-ea19de87ef8e admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:02.774476 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a02df-c0cb-4e10-b3d2-ea19de87ef8e admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:02.775006 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a02df-c0cb-4e10-b3d2-ea19de87ef8e admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:02.776134 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a02df-c0cb-4e10-b3d2-ea19de87ef8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 16:00:02.776573 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a02df-c0cb-4e10-b3d2-ea19de87ef8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:02.778970 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a02df-c0cb-4e10-b3d2-ea19de87ef8e admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:02.794350 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1445/2860] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 16:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 16:00:02.799421 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9247f-173e-4e0e-ad78-f67521aa6f25 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:02.811721 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9247f-173e-4e0e-ad78-f67521aa6f25 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:02.816924 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919fb05-2080-4b5a-90a9-4d84e75b9365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:02.817696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919fb05-2080-4b5a-90a9-4d84e75b9365 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:02.818017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919fb05-2080-4b5a-90a9-4d84e75b9365 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:02.818293 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919fb05-2080-4b5a-90a9-4d84e75b9365 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:02.819011 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919fb05-2080-4b5a-90a9-4d84e75b9365 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:02.821139 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919fb05-2080-4b5a-90a9-4d84e75b9365 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:02.853033 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1416/2861] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 16:00:02 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 16:00:02.857764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a02df-c0cb-4e10-b3d2-ea19de87ef8e admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:02.868525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a02df-c0cb-4e10-b3d2-ea19de87ef8e admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:02.938090 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43fce7-2b61-4bd0-9240-5778d91c3520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:02.938777 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43fce7-2b61-4bd0-9240-5778d91c3520 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:02.939067 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43fce7-2b61-4bd0-9240-5778d91c3520 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:02.939351 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43fce7-2b61-4bd0-9240-5778d91c3520 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:02.940062 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43fce7-2b61-4bd0-9240-5778d91c3520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:02.942061 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43fce7-2b61-4bd0-9240-5778d91c3520 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:02.965807 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1446/2862] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 16:00:02 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 16:00:02.971092 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919fb05-2080-4b5a-90a9-4d84e75b9365 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:02.982310 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919fb05-2080-4b5a-90a9-4d84e75b9365 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:02.987187 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34794cb-a27c-4cc1-b298-21743d8d201c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:02.987937 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34794cb-a27c-4cc1-b298-21743d8d201c admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:02.988162 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34794cb-a27c-4cc1-b298-21743d8d201c admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:02.988448 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34794cb-a27c-4cc1-b298-21743d8d201c admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:02.989186 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34794cb-a27c-4cc1-b298-21743d8d201c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:02.990851 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34794cb-a27c-4cc1-b298-21743d8d201c admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:03.003017 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1417/2863] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 16:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 16:00:03.008311 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43fce7-2b61-4bd0-9240-5778d91c3520 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:03.023200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43fce7-2b61-4bd0-9240-5778d91c3520 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:03.027905 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a112a81-1f3e-4667-a4aa-22bbca450f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:03.028721 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a112a81-1f3e-4667-a4aa-22bbca450f98 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:03.029011 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a112a81-1f3e-4667-a4aa-22bbca450f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:03.029540 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a112a81-1f3e-4667-a4aa-22bbca450f98 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:03.030456 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a112a81-1f3e-4667-a4aa-22bbca450f98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:03.032768 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a112a81-1f3e-4667-a4aa-22bbca450f98 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:03.045483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1447/2864] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 16:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 16:00:03.050210 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34794cb-a27c-4cc1-b298-21743d8d201c admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:03.060570 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34794cb-a27c-4cc1-b298-21743d8d201c admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:03.066270 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46532b-2a40-4dcf-ada1-7706eb58cbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:03.067175 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46532b-2a40-4dcf-ada1-7706eb58cbb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:03.067467 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46532b-2a40-4dcf-ada1-7706eb58cbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:03.067747 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46532b-2a40-4dcf-ada1-7706eb58cbb8 admin admin] PATH_INFO: `/v3/projects/6b111cdd7835462d8734c689972d844c/users/87dfa841300f4385af9ee93a48e09a34/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:03.085510 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46532b-2a40-4dcf-ada1-7706eb58cbb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b111cdd7835462d8734c689972d844c, user_id=87dfa841300f4385af9ee93a48e09a34, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:03.088279 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f46532b-2a40-4dcf-ada1-7706eb58cbb8 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:03.102406 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1418/2865] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 16:00:03 2022] PUT /identity/v3/projects/6b111cdd7835462d8734c689972d844c/users/87dfa841300f4385af9ee93a48e09a34/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 16:00:03.113226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa88d0-8c3c-4066-94c1-c59aede173db None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:03.113521 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa88d0-8c3c-4066-94c1-c59aede173db None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:03.113837 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa88d0-8c3c-4066-94c1-c59aede173db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:03.143079 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-83fa88d0-8c3c-4066-94c1-c59aede173db None None] MFA Rules not processed for user `87dfa841300f4385af9ee93a48e09a34`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 16:00:03.144319 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-83fa88d0-8c3c-4066-94c1-c59aede173db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:03.211960 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1448/2866] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 16:00:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 16:00:03.224856 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46532b-2a40-4dcf-ada1-7706eb58cbb8 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:03.226639 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-9f46532b-2a40-4dcf-ada1-7706eb58cbb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:03.288812 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f46532b-2a40-4dcf-ada1-7706eb58cbb8 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:03.292582 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada08e52-a569-4e1b-b7b6-43b826080e0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51404afd35b94c47ba5c44f0deb73e30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:03.293243 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada08e52-a569-4e1b-b7b6-43b826080e0f service cinder] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:03.293528 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada08e52-a569-4e1b-b7b6-43b826080e0f service cinder] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:03.293844 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada08e52-a569-4e1b-b7b6-43b826080e0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:03.304054 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada08e52-a569-4e1b-b7b6-43b826080e0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:03.305555 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada08e52-a569-4e1b-b7b6-43b826080e0f service cinder] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:03.316193 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1419/2867] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 16:00:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 16:00:03.343494 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa88d0-8c3c-4066-94c1-c59aede173db None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:03.408262 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa88d0-8c3c-4066-94c1-c59aede173db None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:03.412788 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cfa8c-f76a-4402-a23b-84f2ccc3397f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:03.413677 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cfa8c-f76a-4402-a23b-84f2ccc3397f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:03.413964 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cfa8c-f76a-4402-a23b-84f2ccc3397f admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:03.414240 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cfa8c-f76a-4402-a23b-84f2ccc3397f admin admin] PATH_INFO: `/v3/users/87dfa841300f4385af9ee93a48e09a34` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:03.415442 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180cfa8c-f76a-4402-a23b-84f2ccc3397f admin admin] RBAC: Authorizing `identity:delete_user(user_id=87dfa841300f4385af9ee93a48e09a34)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:03.417346 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180cfa8c-f76a-4402-a23b-84f2ccc3397f admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:03.419060 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:00:03.471572 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-180cfa8c-f76a-4402-a23b-84f2ccc3397f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87dfa841300f4385af9ee93a48e09a34'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:03.484800 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-180cfa8c-f76a-4402-a23b-84f2ccc3397f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87dfa841300f4385af9ee93a48e09a34'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:03.485583 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-180cfa8c-f76a-4402-a23b-84f2ccc3397f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87dfa841300f4385af9ee93a48e09a34'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:03.494738 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-180cfa8c-f76a-4402-a23b-84f2ccc3397f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87dfa841300f4385af9ee93a48e09a34'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:03.505896 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-180cfa8c-f76a-4402-a23b-84f2ccc3397f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87dfa841300f4385af9ee93a48e09a34'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:03.512236 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1449/2868] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 16:00:03 2022] DELETE /identity/v3/users/87dfa841300f4385af9ee93a48e09a34 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 16:00:03.518176 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639a0a1-d29b-4093-92af-5156c38cab54 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:03.518607 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639a0a1-d29b-4093-92af-5156c38cab54 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:03.519004 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639a0a1-d29b-4093-92af-5156c38cab54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:03.526272 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:00:03.541407 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-1639a0a1-d29b-4093-92af-5156c38cab54 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 16:00:03.542583 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1639a0a1-d29b-4093-92af-5156c38cab54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:03.611541 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1420/2869] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 16:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 16:00:03.621361 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cfa8c-f76a-4402-a23b-84f2ccc3397f admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:03.622996 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-180cfa8c-f76a-4402-a23b-84f2ccc3397f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:03.686988 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cfa8c-f76a-4402-a23b-84f2ccc3397f admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:03.691663 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7754dda-9c4d-44af-a559-09213e715ea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:03.692378 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7754dda-9c4d-44af-a559-09213e715ea8 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:03.692670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7754dda-9c4d-44af-a559-09213e715ea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:03.692946 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7754dda-9c4d-44af-a559-09213e715ea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:03.703670 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7754dda-9c4d-44af-a559-09213e715ea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:03.705132 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7754dda-9c4d-44af-a559-09213e715ea8 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:03.716082 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1450/2870] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 16:00:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 16:00:04.017610 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639a0a1-d29b-4093-92af-5156c38cab54 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:04.019442 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-1639a0a1-d29b-4093-92af-5156c38cab54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:04.042085 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639a0a1-d29b-4093-92af-5156c38cab54 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:04.053613 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24892d-644d-4a2e-8a3a-cf767d2c0574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:04.054719 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24892d-644d-4a2e-8a3a-cf767d2c0574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:04.055122 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24892d-644d-4a2e-8a3a-cf767d2c0574 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:04.055535 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24892d-644d-4a2e-8a3a-cf767d2c0574 admin admin] PATH_INFO: `/v3/projects/6b111cdd7835462d8734c689972d844c` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:04.056944 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24892d-644d-4a2e-8a3a-cf767d2c0574 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b111cdd7835462d8734c689972d844c)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:04.059809 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24892d-644d-4a2e-8a3a-cf767d2c0574 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:04.119614 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-8d24892d-644d-4a2e-8a3a-cf767d2c0574 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b111cdd7835462d8734c689972d844c', 'request_id': 'req-8d24892d-644d-4a2e-8a3a-cf767d2c0574'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:04.127204 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-8d24892d-644d-4a2e-8a3a-cf767d2c0574 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b111cdd7835462d8734c689972d844c', 'request_id': 'req-8d24892d-644d-4a2e-8a3a-cf767d2c0574'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:04.133842 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-8d24892d-644d-4a2e-8a3a-cf767d2c0574 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b111cdd7835462d8734c689972d844c', 'request_id': 'req-8d24892d-644d-4a2e-8a3a-cf767d2c0574'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:04.149373 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-8d24892d-644d-4a2e-8a3a-cf767d2c0574 admin admin] The token cache is being invalidate because project 6b111cdd7835462d8734c689972d844c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 16:00:04.149697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-8d24892d-644d-4a2e-8a3a-cf767d2c0574 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:04.150419 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-8d24892d-644d-4a2e-8a3a-cf767d2c0574 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:00:04.152046 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1421/2871] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 16:00:04 2022] DELETE /identity/v3/projects/6b111cdd7835462d8734c689972d844c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 16:00:04.158187 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772f66f-0d79-47f3-9e23-00637c02f843 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:04.158480 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772f66f-0d79-47f3-9e23-00637c02f843 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:04.158764 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772f66f-0d79-47f3-9e23-00637c02f843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:04.178977 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-b772f66f-0d79-47f3-9e23-00637c02f843 None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 16:00:04.180233 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-b772f66f-0d79-47f3-9e23-00637c02f843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:04.254651 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1451/2872] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 16:00:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 16:00:04.264311 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b20ba6-6865-4ad5-a268-9cbe66e24eaf None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:04.264650 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b20ba6-6865-4ad5-a268-9cbe66e24eaf None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:04.264979 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b20ba6-6865-4ad5-a268-9cbe66e24eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:04.286732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-a5b20ba6-6865-4ad5-a268-9cbe66e24eaf None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 16:00:04.287971 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-a5b20ba6-6865-4ad5-a268-9cbe66e24eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:04.296356 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1422/2873] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 16:00:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 16:00:04.302619 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772f66f-0d79-47f3-9e23-00637c02f843 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:04.318469 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772f66f-0d79-47f3-9e23-00637c02f843 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:04.323368 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c626e-cf9d-41fc-ac25-6545d0442aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:04.324047 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c626e-cf9d-41fc-ac25-6545d0442aab admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:04.324340 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c626e-cf9d-41fc-ac25-6545d0442aab admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:04.324617 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c626e-cf9d-41fc-ac25-6545d0442aab admin admin] PATH_INFO: `/v3/domains` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:04.325366 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c626e-cf9d-41fc-ac25-6545d0442aab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91942) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 16:00:04.325700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c626e-cf9d-41fc-ac25-6545d0442aab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:04.327331 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c626e-cf9d-41fc-ac25-6545d0442aab admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:04.340717 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1452/2874] 10.222.0.20 () {58 vars in 1152 bytes} [Mon Jun 27 16:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 16:00:04.346020 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b20ba6-6865-4ad5-a268-9cbe66e24eaf None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:04.357730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b20ba6-6865-4ad5-a268-9cbe66e24eaf None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:04.364039 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa08573-521f-41c4-960d-a3912e63af95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:04.364727 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa08573-521f-41c4-960d-a3912e63af95 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:04.365045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa08573-521f-41c4-960d-a3912e63af95 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:04.365325 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa08573-521f-41c4-960d-a3912e63af95 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:04.366095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa08573-521f-41c4-960d-a3912e63af95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:04.368039 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa08573-521f-41c4-960d-a3912e63af95 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:04.401696 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1423/2875] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 16:00:04 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 16:00:04.406506 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c626e-cf9d-41fc-ac25-6545d0442aab admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:04.416838 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c626e-cf9d-41fc-ac25-6545d0442aab admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:04.423150 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:00:04.480208 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e082d69-9d4c-4c00-bdab-0648d9a366d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:04.480898 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e082d69-9d4c-4c00-bdab-0648d9a366d8 admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:04.481192 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e082d69-9d4c-4c00-bdab-0648d9a366d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:04.481471 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e082d69-9d4c-4c00-bdab-0648d9a366d8 admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:04.482221 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e082d69-9d4c-4c00-bdab-0648d9a366d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:04.484710 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e082d69-9d4c-4c00-bdab-0648d9a366d8 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:04.505349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1453/2876] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 16:00:04 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 16:00:04.509569 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa08573-521f-41c4-960d-a3912e63af95 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:04.520101 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa08573-521f-41c4-960d-a3912e63af95 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:04.524722 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9e63a-38ed-4371-8b76-002b573c7b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:04.525387 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9e63a-38ed-4371-8b76-002b573c7b67 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:04.525697 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9e63a-38ed-4371-8b76-002b573c7b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:04.525976 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9e63a-38ed-4371-8b76-002b573c7b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:04.526712 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9e63a-38ed-4371-8b76-002b573c7b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:04.528291 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9e63a-38ed-4371-8b76-002b573c7b67 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:04.539609 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1424/2877] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 16:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 16:00:04.543690 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e082d69-9d4c-4c00-bdab-0648d9a366d8 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:04.555904 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e082d69-9d4c-4c00-bdab-0648d9a366d8 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:04.561951 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d595a71-2dcd-42a0-a1a9-d3530309ff20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:04.562839 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d595a71-2dcd-42a0-a1a9-d3530309ff20 admin admin] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:04.563242 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d595a71-2dcd-42a0-a1a9-d3530309ff20 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:04.563630 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d595a71-2dcd-42a0-a1a9-d3530309ff20 admin admin] PATH_INFO: `/v3/roles` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:04.564617 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d595a71-2dcd-42a0-a1a9-d3530309ff20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:04.566463 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d595a71-2dcd-42a0-a1a9-d3530309ff20 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:04.578358 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1454/2878] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 16:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 16:00:04.582686 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9e63a-38ed-4371-8b76-002b573c7b67 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:04.595301 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9e63a-38ed-4371-8b76-002b573c7b67 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:04.602747 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396a82c-7833-4a7d-9fb9-f5d84fa9b3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:04.603654 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396a82c-7833-4a7d-9fb9-f5d84fa9b3a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:04.603941 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396a82c-7833-4a7d-9fb9-f5d84fa9b3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:04.604218 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396a82c-7833-4a7d-9fb9-f5d84fa9b3a6 admin admin] PATH_INFO: `/v3/projects/636c72ef28514ac5aef5779a137d9df7/users/bffd1a8597e94f69b1bd98bd6ea7b294/roles/5dc71eded4f64c08b2fb5f545b63b2f6` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:04.620164 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c396a82c-7833-4a7d-9fb9-f5d84fa9b3a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=636c72ef28514ac5aef5779a137d9df7, user_id=bffd1a8597e94f69b1bd98bd6ea7b294, role_id=5dc71eded4f64c08b2fb5f545b63b2f6)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:04.623165 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c396a82c-7833-4a7d-9fb9-f5d84fa9b3a6 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:04.638006 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1425/2879] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 16:00:04 2022] PUT /identity/v3/projects/636c72ef28514ac5aef5779a137d9df7/users/bffd1a8597e94f69b1bd98bd6ea7b294/roles/5dc71eded4f64c08b2fb5f545b63b2f6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 16:00:04.644315 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898fae29-1658-4c53-9f4d-9e01a52ed13f None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:04.644815 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898fae29-1658-4c53-9f4d-9e01a52ed13f None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:04.644956 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898fae29-1658-4c53-9f4d-9e01a52ed13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:04.664353 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-898fae29-1658-4c53-9f4d-9e01a52ed13f None None] MFA Rules not processed for user `4b433ad0233c4e5dbe8fad9896962e25`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 16:00:04.665521 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-898fae29-1658-4c53-9f4d-9e01a52ed13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:04.728334 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1455/2880] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 16:00:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 16:00:04.738041 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396a82c-7833-4a7d-9fb9-f5d84fa9b3a6 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:04.739735 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-c396a82c-7833-4a7d-9fb9-f5d84fa9b3a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:04.769952 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:00:04.802256 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396a82c-7833-4a7d-9fb9-f5d84fa9b3a6 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:04.805928 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a68f91-80cd-4cdf-b477-09d85d9b796f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:04.806583 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a68f91-80cd-4cdf-b477-09d85d9b796f service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:04.806865 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a68f91-80cd-4cdf-b477-09d85d9b796f service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:04.807139 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a68f91-80cd-4cdf-b477-09d85d9b796f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:04.817183 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a68f91-80cd-4cdf-b477-09d85d9b796f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:04.818648 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a68f91-80cd-4cdf-b477-09d85d9b796f service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:04.830766 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1426/2881] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 16:00:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 16:00:10.063709 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec8393-74d1-4a74-9980-86622d3cd785 None None] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:10.065781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec8393-74d1-4a74-9980-86622d3cd785 None None] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:10.065781 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec8393-74d1-4a74-9980-86622d3cd785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:10.099435 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.auth.core [None req-fcec8393-74d1-4a74-9980-86622d3cd785 None None] MFA Rules not processed for user `bffd1a8597e94f69b1bd98bd6ea7b294`. Rule list: `[]` (Enabled: `True`). {{(pid=91942) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 16:00:10.102110 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-fcec8393-74d1-4a74-9980-86622d3cd785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:10.192267 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1456/2882] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 16:00:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 16:00:10.197479 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a68f91-80cd-4cdf-b477-09d85d9b796f service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:10.210513 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a68f91-80cd-4cdf-b477-09d85d9b796f service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:10.215957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981dbf6-60e7-4cbc-9852-9d95b06d3ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:10.216645 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981dbf6-60e7-4cbc-9852-9d95b06d3ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:10.216929 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981dbf6-60e7-4cbc-9852-9d95b06d3ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:10.217212 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0981dbf6-60e7-4cbc-9852-9d95b06d3ad0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:10.217957 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0981dbf6-60e7-4cbc-9852-9d95b06d3ad0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:10.220497 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0981dbf6-60e7-4cbc-9852-9d95b06d3ad0 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:10.250348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1427/2883] 10.222.0.20 () {60 vars in 1152 bytes} [Mon Jun 27 16:00:10 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 16:00:10.257156 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec8393-74d1-4a74-9980-86622d3cd785 None None] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:10.270651 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec8393-74d1-4a74-9980-86622d3cd785 None None] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:10.291669 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:00:10.361093 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6be5c-61c5-45c5-944d-0228e1c3503f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:10.361991 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6be5c-61c5-45c5-944d-0228e1c3503f admin admin] REQUEST_METHOD: `POST` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:10.362355 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6be5c-61c5-45c5-944d-0228e1c3503f admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:10.362712 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6be5c-61c5-45c5-944d-0228e1c3503f admin admin] PATH_INFO: `/v3/users` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:10.363497 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6be5c-61c5-45c5-944d-0228e1c3503f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:10.365576 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6be5c-61c5-45c5-944d-0228e1c3503f admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:10.389325 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1457/2884] 10.222.0.20 () {60 vars in 1143 bytes} [Mon Jun 27 16:00:10 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 16:00:10.394362 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981dbf6-60e7-4cbc-9852-9d95b06d3ad0 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:10.405466 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0981dbf6-60e7-4cbc-9852-9d95b06d3ad0 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:10.410626 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3137e0-0b8b-47d4-9ef5-bf45299580fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:10.411318 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3137e0-0b8b-47d4-9ef5-bf45299580fc admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:10.411606 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3137e0-0b8b-47d4-9ef5-bf45299580fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:10.411884 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3137e0-0b8b-47d4-9ef5-bf45299580fc admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:10.412635 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3137e0-0b8b-47d4-9ef5-bf45299580fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:10.414310 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3137e0-0b8b-47d4-9ef5-bf45299580fc admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:10.426854 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1428/2885] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 16:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 16:00:10.431727 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6be5c-61c5-45c5-944d-0228e1c3503f admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:10.444096 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6be5c-61c5-45c5-944d-0228e1c3503f admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:10.449120 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498b0a0-f896-4ea6-a60a-ac4bfad9378d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:10.450078 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498b0a0-f896-4ea6-a60a-ac4bfad9378d admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:10.450367 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498b0a0-f896-4ea6-a60a-ac4bfad9378d admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:10.450646 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498b0a0-f896-4ea6-a60a-ac4bfad9378d admin admin] PATH_INFO: `/v3/projects/c4f78273b11a4d5c84d1cf4ad58aa99c/users/74b377804f1940ec85ffc445e191306a/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:10.475550 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498b0a0-f896-4ea6-a60a-ac4bfad9378d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f78273b11a4d5c84d1cf4ad58aa99c, user_id=74b377804f1940ec85ffc445e191306a, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:10.479464 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498b0a0-f896-4ea6-a60a-ac4bfad9378d admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:10.497151 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1458/2886] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 16:00:10 2022] PUT /identity/v3/projects/c4f78273b11a4d5c84d1cf4ad58aa99c/users/74b377804f1940ec85ffc445e191306a/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 16:00:10.503729 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3137e0-0b8b-47d4-9ef5-bf45299580fc admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:10.516795 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3137e0-0b8b-47d4-9ef5-bf45299580fc admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:10.589071 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb76ced-6b79-4fff-b193-7c134c6dfd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:10.589801 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb76ced-6b79-4fff-b193-7c134c6dfd46 admin admin] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:10.590090 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb76ced-6b79-4fff-b193-7c134c6dfd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:10.590373 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb76ced-6b79-4fff-b193-7c134c6dfd46 admin admin] PATH_INFO: `/v3/roles` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:10.591144 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb76ced-6b79-4fff-b193-7c134c6dfd46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:10.592787 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb76ced-6b79-4fff-b193-7c134c6dfd46 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:10.605197 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1429/2887] 10.222.0.20 () {58 vars in 1121 bytes} [Mon Jun 27 16:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 16:00:10.610958 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498b0a0-f896-4ea6-a60a-ac4bfad9378d admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:10.628831 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498b0a0-f896-4ea6-a60a-ac4bfad9378d admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:10.635347 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f59a9-bb3a-4210-b24a-f526807685f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:10.636578 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f59a9-bb3a-4210-b24a-f526807685f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:10.636893 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f59a9-bb3a-4210-b24a-f526807685f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:10.637357 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f59a9-bb3a-4210-b24a-f526807685f7 admin admin] PATH_INFO: `/v3/projects/c4f78273b11a4d5c84d1cf4ad58aa99c/users/74b377804f1940ec85ffc445e191306a/roles/13962c72b72c40ba9bf480db87b2dcfa` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:10.640927 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f59a9-bb3a-4210-b24a-f526807685f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f78273b11a4d5c84d1cf4ad58aa99c, user_id=74b377804f1940ec85ffc445e191306a, role_id=13962c72b72c40ba9bf480db87b2dcfa)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:10.643701 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f59a9-bb3a-4210-b24a-f526807685f7 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:10.655571 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1459/2888] 10.222.0.20 () {60 vars in 1482 bytes} [Mon Jun 27 16:00:10 2022] PUT /identity/v3/projects/c4f78273b11a4d5c84d1cf4ad58aa99c/users/74b377804f1940ec85ffc445e191306a/roles/13962c72b72c40ba9bf480db87b2dcfa => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 16:00:16.177329 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93635537-d61e-450b-bd73-32094c9e3245 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:16.177874 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93635537-d61e-450b-bd73-32094c9e3245 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:16.177963 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93635537-d61e-450b-bd73-32094c9e3245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:16.227493 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-93635537-d61e-450b-bd73-32094c9e3245 None None] MFA Rules not processed for user `74b377804f1940ec85ffc445e191306a`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 16:00:16.229314 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-93635537-d61e-450b-bd73-32094c9e3245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:16.340714 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:00:16.385496 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1430/2889] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 16:00:16 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 16:00:16.408268 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f59a9-bb3a-4210-b24a-f526807685f7 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:16.413018 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-d64f59a9-bb3a-4210-b24a-f526807685f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:16.738747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f59a9-bb3a-4210-b24a-f526807685f7 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:16.744827 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5571112-4a2b-4501-b7c2-dd53c4f798ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:16.745698 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5571112-4a2b-4501-b7c2-dd53c4f798ef service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:16.745995 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5571112-4a2b-4501-b7c2-dd53c4f798ef service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:16.746279 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5571112-4a2b-4501-b7c2-dd53c4f798ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:16.791398 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:00:16.845818 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5571112-4a2b-4501-b7c2-dd53c4f798ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:16.847450 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5571112-4a2b-4501-b7c2-dd53c4f798ef service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:16.858071 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1460/2890] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 16:00:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 16:00:18.197045 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fedf4d-e609-48a5-a838-c800d90387e7 None None] REQUEST_METHOD: `POST` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:18.198002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fedf4d-e609-48a5-a838-c800d90387e7 None None] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:18.198002 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fedf4d-e609-48a5-a838-c800d90387e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:18.216678 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.auth.core [None req-48fedf4d-e609-48a5-a838-c800d90387e7 None None] MFA Rules not processed for user `bffd1a8597e94f69b1bd98bd6ea7b294`. Rule list: `[]` (Enabled: `True`). {{(pid=91941) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 16:00:18.218015 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-48fedf4d-e609-48a5-a838-c800d90387e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:18.225730 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1431/2891] 10.222.0.20 () {58 vars in 957 bytes} [Mon Jun 27 16:00:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 16:00:26.853257 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=15 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:00:28.210983 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=11 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:00:32.958496 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5571112-4a2b-4501-b7c2-dd53c4f798ef service nova] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:32.960002 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-c5571112-4a2b-4501-b7c2-dd53c4f798ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:33.041335 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5571112-4a2b-4501-b7c2-dd53c4f798ef service nova] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:33.045654 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b596a1-c906-4021-8786-4f3a0163fea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:33.047104 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b596a1-c906-4021-8786-4f3a0163fea5 service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:33.047245 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b596a1-c906-4021-8786-4f3a0163fea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:33.047475 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b596a1-c906-4021-8786-4f3a0163fea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:33.049447 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-24b596a1-c906-4021-8786-4f3a0163fea5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:33.059748 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b596a1-c906-4021-8786-4f3a0163fea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:33.061193 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b596a1-c906-4021-8786-4f3a0163fea5 service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:33.070026 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1461/2892] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 16:00:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 16:00:33.580258 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fedf4d-e609-48a5-a838-c800d90387e7 None None] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:33.581784 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-48fedf4d-e609-48a5-a838-c800d90387e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:33.662038 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fedf4d-e609-48a5-a838-c800d90387e7 None None] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:33.667316 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a057c3-39d5-436c-a097-9cb1478ceb9c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffafb68f1bb941588a5217ad83ad907c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:33.667990 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a057c3-39d5-436c-a097-9cb1478ceb9c service placement] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:33.668282 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a057c3-39d5-436c-a097-9cb1478ceb9c service placement] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:33.668568 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a057c3-39d5-436c-a097-9cb1478ceb9c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:33.670451 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-87a057c3-39d5-436c-a097-9cb1478ceb9c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:33.684190 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a057c3-39d5-436c-a097-9cb1478ceb9c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:33.685831 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a057c3-39d5-436c-a097-9cb1478ceb9c service placement] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:33.696819 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1432/2893] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 16:00:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 16:00:43.066293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=35 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:00:43.692110 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=32 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:00:47.217006 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b596a1-c906-4021-8786-4f3a0163fea5 service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:47.218638 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-24b596a1-c906-4021-8786-4f3a0163fea5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:47.237500 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b596a1-c906-4021-8786-4f3a0163fea5 service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:47.243180 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37e648-9ddd-45ff-a1e3-f1c38cf7300d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:47.243835 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37e648-9ddd-45ff-a1e3-f1c38cf7300d service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:47.244121 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37e648-9ddd-45ff-a1e3-f1c38cf7300d service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:47.244396 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37e648-9ddd-45ff-a1e3-f1c38cf7300d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:47.246202 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-bd37e648-9ddd-45ff-a1e3-f1c38cf7300d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:00:47.260742 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37e648-9ddd-45ff-a1e3-f1c38cf7300d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:47.262373 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37e648-9ddd-45ff-a1e3-f1c38cf7300d service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:47.275317 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1462/2894] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 16:00:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 16:00:49.180993 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a057c3-39d5-436c-a097-9cb1478ceb9c service placement] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:00:49.191820 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a057c3-39d5-436c-a097-9cb1478ceb9c service placement] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:00:49.195498 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede176e1-23a3-47c6-a2df-e04a2aaecef8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:00:49.196296 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede176e1-23a3-47c6-a2df-e04a2aaecef8 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:00:49.197094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede176e1-23a3-47c6-a2df-e04a2aaecef8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:00:49.197094 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede176e1-23a3-47c6-a2df-e04a2aaecef8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:00:49.207085 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede176e1-23a3-47c6-a2df-e04a2aaecef8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:00:49.208571 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede176e1-23a3-47c6-a2df-e04a2aaecef8 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:00:49.219894 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1433/2895] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 16:00:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 16:00:57.269726 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=6 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:00:59.214366 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=6 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:02:04.633863 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37e648-9ddd-45ff-a1e3-f1c38cf7300d service nova] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:02:04.635261 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-bd37e648-9ddd-45ff-a1e3-f1c38cf7300d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:02:04.671124 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37e648-9ddd-45ff-a1e3-f1c38cf7300d service nova] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:02:04.677296 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522c8ab-e6de-4651-a9ce-901ccce08690 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffafb68f1bb941588a5217ad83ad907c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:02:04.677963 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522c8ab-e6de-4651-a9ce-901ccce08690 service placement] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:02:04.678242 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522c8ab-e6de-4651-a9ce-901ccce08690 service placement] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:02:04.678517 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522c8ab-e6de-4651-a9ce-901ccce08690 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:02:04.680265 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-2522c8ab-e6de-4651-a9ce-901ccce08690 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:02:04.692351 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522c8ab-e6de-4651-a9ce-901ccce08690 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:02:04.693937 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522c8ab-e6de-4651-a9ce-901ccce08690 service placement] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:02:04.703747 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1463/2896] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 16:02:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 16:02:14.699525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=15 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:03:00.091860 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede176e1-23a3-47c6-a2df-e04a2aaecef8 service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:03:00.101576 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede176e1-23a3-47c6-a2df-e04a2aaecef8 service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:03:00.105317 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d14ace-e5e3-4953-8b62-e8fb3f57a4d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:03:00.105993 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d14ace-e5e3-4953-8b62-e8fb3f57a4d7 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:03:00.106440 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d14ace-e5e3-4953-8b62-e8fb3f57a4d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:03:00.106723 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d14ace-e5e3-4953-8b62-e8fb3f57a4d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:03:00.108576 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-68d14ace-e5e3-4953-8b62-e8fb3f57a4d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:03:00.111170 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522c8ab-e6de-4651-a9ce-901ccce08690 service placement] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:03:00.119917 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d14ace-e5e3-4953-8b62-e8fb3f57a4d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:03:00.121140 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522c8ab-e6de-4651-a9ce-901ccce08690 service placement] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:03:00.121732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d14ace-e5e3-4953-8b62-e8fb3f57a4d7 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:03:00.124805 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09014a93-3034-467c-aa7b-e6af6234236c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:03:00.125447 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09014a93-3034-467c-aa7b-e6af6234236c service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:03:00.125773 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09014a93-3034-467c-aa7b-e6af6234236c service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:03:00.126042 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09014a93-3034-467c-aa7b-e6af6234236c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:03:00.127857 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-09014a93-3034-467c-aa7b-e6af6234236c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:03:00.131912 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1434/2897] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 16:03:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 16:03:00.163169 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09014a93-3034-467c-aa7b-e6af6234236c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:03:00.165083 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09014a93-3034-467c-aa7b-e6af6234236c service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:03:00.188807 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1464/2898] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 16:03:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 16:03:01.225703 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d14ace-e5e3-4953-8b62-e8fb3f57a4d7 service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:03:01.234441 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d14ace-e5e3-4953-8b62-e8fb3f57a4d7 service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:03:01.238113 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48ff06-3144-467d-9588-389a1a674c9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:03:01.238754 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48ff06-3144-467d-9588-389a1a674c9b service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:03:01.239039 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48ff06-3144-467d-9588-389a1a674c9b service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:03:01.239324 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48ff06-3144-467d-9588-389a1a674c9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:03:01.241699 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-dc48ff06-3144-467d-9588-389a1a674c9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:03:01.256814 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48ff06-3144-467d-9588-389a1a674c9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:03:01.258284 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48ff06-3144-467d-9588-389a1a674c9b service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:03:01.270042 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1435/2899] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 16:03:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 16:03:01.364339 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09014a93-3034-467c-aa7b-e6af6234236c service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:03:01.374038 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09014a93-3034-467c-aa7b-e6af6234236c service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:03:01.378048 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45466a1-8b55-4bb6-837a-4451c125984a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:03:01.378727 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45466a1-8b55-4bb6-837a-4451c125984a service neutron] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:03:01.379013 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45466a1-8b55-4bb6-837a-4451c125984a service neutron] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:03:01.379563 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45466a1-8b55-4bb6-837a-4451c125984a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:03:01.381846 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-d45466a1-8b55-4bb6-837a-4451c125984a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:03:01.395917 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45466a1-8b55-4bb6-837a-4451c125984a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:03:01.397921 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45466a1-8b55-4bb6-837a-4451c125984a service neutron] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:03:01.409483 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1465/2900] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 16:03:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 16:03:11.264907 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=12 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:03:11.404270 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=24 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:03:18.202422 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48ff06-3144-467d-9588-389a1a674c9b service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:03:18.211315 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48ff06-3144-467d-9588-389a1a674c9b service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:03:18.215663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fde985-0ee5-41a2-b8f7-3734ea0c0ac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42dee639e2834d99b7468f911c37cf8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:03:18.216321 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fde985-0ee5-41a2-b8f7-3734ea0c0ac2 service neutron] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:03:18.216603 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fde985-0ee5-41a2-b8f7-3734ea0c0ac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:03:18.216878 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fde985-0ee5-41a2-b8f7-3734ea0c0ac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:03:18.218724 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-47fde985-0ee5-41a2-b8f7-3734ea0c0ac2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:03:18.230868 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fde985-0ee5-41a2-b8f7-3734ea0c0ac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:03:18.232322 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fde985-0ee5-41a2-b8f7-3734ea0c0ac2 service neutron] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:03:18.241034 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1436/2901] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 16:03:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 16:03:28.237133 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=6 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:04:33.608044 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45466a1-8b55-4bb6-837a-4451c125984a service neutron] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:04:33.621136 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45466a1-8b55-4bb6-837a-4451c125984a service neutron] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:04:33.628585 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1cba9-2f7a-4fc3-8539-5a56f2557e5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c8f6262cf24419daf803ae71aed24f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:04:33.628798 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1cba9-2f7a-4fc3-8539-5a56f2557e5c service nova] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:04:33.629087 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1cba9-2f7a-4fc3-8539-5a56f2557e5c service nova] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:04:33.629367 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1cba9-2f7a-4fc3-8539-5a56f2557e5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:04:33.631269 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-59e1cba9-2f7a-4fc3-8539-5a56f2557e5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:04:33.644914 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e1cba9-2f7a-4fc3-8539-5a56f2557e5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:04:33.646525 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e1cba9-2f7a-4fc3-8539-5a56f2557e5c service nova] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:04:33.659190 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1466/2902] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 16:04:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 16:04:40.300856 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fde985-0ee5-41a2-b8f7-3734ea0c0ac2 service neutron] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:04:40.318171 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fde985-0ee5-41a2-b8f7-3734ea0c0ac2 service neutron] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:04:40.323838 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b5851-320f-4dfc-9722-6a983b63c661 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffafb68f1bb941588a5217ad83ad907c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:04:40.324505 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b5851-320f-4dfc-9722-6a983b63c661 service placement] REQUEST_METHOD: `GET` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:04:40.324792 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b5851-320f-4dfc-9722-6a983b63c661 service placement] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:04:40.325068 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b5851-320f-4dfc-9722-6a983b63c661 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:04:40.327121 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-6e5b5851-320f-4dfc-9722-6a983b63c661 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:04:40.340855 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b5851-320f-4dfc-9722-6a983b63c661 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:04:40.342813 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b5851-320f-4dfc-9722-6a983b63c661 service placement] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:04:40.356808 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1437/2903] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 16:04:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 16:04:43.654794 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=6 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:04:43.951199 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1cba9-2f7a-4fc3-8539-5a56f2557e5c service nova] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:04:44.037297 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1cba9-2f7a-4fc3-8539-5a56f2557e5c service nova] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:04:44.042565 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ece8b-c52d-4b21-9011-ae36a1e02a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:04:44.043433 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ece8b-c52d-4b21-9011-ae36a1e02a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:04:44.043712 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ece8b-c52d-4b21-9011-ae36a1e02a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:04:44.043996 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ece8b-c52d-4b21-9011-ae36a1e02a66 admin admin] PATH_INFO: `/v3/users/bffd1a8597e94f69b1bd98bd6ea7b294` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:04:44.045172 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ece8b-c52d-4b21-9011-ae36a1e02a66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bffd1a8597e94f69b1bd98bd6ea7b294)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:04:44.047349 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ece8b-c52d-4b21-9011-ae36a1e02a66 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:04:44.096603 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-691ece8b-c52d-4b21-9011-ae36a1e02a66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bffd1a8597e94f69b1bd98bd6ea7b294'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:44.110581 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-691ece8b-c52d-4b21-9011-ae36a1e02a66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bffd1a8597e94f69b1bd98bd6ea7b294'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:44.111244 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-691ece8b-c52d-4b21-9011-ae36a1e02a66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bffd1a8597e94f69b1bd98bd6ea7b294'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:44.119700 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-691ece8b-c52d-4b21-9011-ae36a1e02a66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bffd1a8597e94f69b1bd98bd6ea7b294'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:44.128435 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-691ece8b-c52d-4b21-9011-ae36a1e02a66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bffd1a8597e94f69b1bd98bd6ea7b294'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:44.134841 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1467/2904] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 16:04:43 2022] DELETE /identity/v3/users/bffd1a8597e94f69b1bd98bd6ea7b294 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 16:04:44.340208 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b5851-320f-4dfc-9722-6a983b63c661 service placement] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:04:44.355182 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b5851-320f-4dfc-9722-6a983b63c661 service placement] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:04:44.433539 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20a060-b6b2-4724-b74b-5798e7d1ae21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:04:44.434454 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20a060-b6b2-4724-b74b-5798e7d1ae21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:04:44.434732 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20a060-b6b2-4724-b74b-5798e7d1ae21 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:04:44.435003 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20a060-b6b2-4724-b74b-5798e7d1ae21 admin admin] PATH_INFO: `/v3/projects/636c72ef28514ac5aef5779a137d9df7` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:04:44.436255 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20a060-b6b2-4724-b74b-5798e7d1ae21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=636c72ef28514ac5aef5779a137d9df7)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:04:44.438242 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20a060-b6b2-4724-b74b-5798e7d1ae21 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:04:44.494015 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4e20a060-b6b2-4724-b74b-5798e7d1ae21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '636c72ef28514ac5aef5779a137d9df7', 'request_id': 'req-4e20a060-b6b2-4724-b74b-5798e7d1ae21'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:44.500763 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4e20a060-b6b2-4724-b74b-5798e7d1ae21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '636c72ef28514ac5aef5779a137d9df7', 'request_id': 'req-4e20a060-b6b2-4724-b74b-5798e7d1ae21'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:44.508663 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4e20a060-b6b2-4724-b74b-5798e7d1ae21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '636c72ef28514ac5aef5779a137d9df7', 'request_id': 'req-4e20a060-b6b2-4724-b74b-5798e7d1ae21'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:44.535165 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4e20a060-b6b2-4724-b74b-5798e7d1ae21 admin admin] The token cache is being invalidate because project 636c72ef28514ac5aef5779a137d9df7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 16:04:44.535453 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4e20a060-b6b2-4724-b74b-5798e7d1ae21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:44.536093 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-4e20a060-b6b2-4724-b74b-5798e7d1ae21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:44.537731 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1438/2905] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 16:04:44 2022] DELETE /identity/v3/projects/636c72ef28514ac5aef5779a137d9df7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 16:04:44.542401 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ece8b-c52d-4b21-9011-ae36a1e02a66 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:04:44.543903 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-691ece8b-c52d-4b21-9011-ae36a1e02a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:04:44.557884 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ece8b-c52d-4b21-9011-ae36a1e02a66 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:04:44.618809 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e89a6-c396-4cc1-9154-87b999993273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:04:44.619833 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e89a6-c396-4cc1-9154-87b999993273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:04:44.620200 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e89a6-c396-4cc1-9154-87b999993273 admin admin] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:04:44.620551 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e89a6-c396-4cc1-9154-87b999993273 admin admin] PATH_INFO: `/v3/users/74b377804f1940ec85ffc445e191306a` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:04:44.621827 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e89a6-c396-4cc1-9154-87b999993273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74b377804f1940ec85ffc445e191306a)` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:04:44.623923 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e89a6-c396-4cc1-9154-87b999993273 admin admin] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:04:44.626741 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:04:44.678406 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8f8e89a6-c396-4cc1-9154-87b999993273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74b377804f1940ec85ffc445e191306a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:44.690226 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8f8e89a6-c396-4cc1-9154-87b999993273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74b377804f1940ec85ffc445e191306a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:44.690910 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8f8e89a6-c396-4cc1-9154-87b999993273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74b377804f1940ec85ffc445e191306a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:44.698622 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8f8e89a6-c396-4cc1-9154-87b999993273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74b377804f1940ec85ffc445e191306a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:44.705672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.notifications [None req-8f8e89a6-c396-4cc1-9154-87b999993273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74b377804f1940ec85ffc445e191306a'} {{(pid=91942) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:44.712293 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1468/2906] 10.222.0.20 () {58 vars in 1223 bytes} [Mon Jun 27 16:04:44 2022] DELETE /identity/v3/users/74b377804f1940ec85ffc445e191306a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 16:04:44.894588 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20a060-b6b2-4724-b74b-5798e7d1ae21 admin admin] Authenticating user token {{(pid=91941) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:04:44.896527 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.fernet_utils [None req-4e20a060-b6b2-4724-b74b-5798e7d1ae21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91941) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:04:44.910536 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20a060-b6b2-4724-b74b-5798e7d1ae21 admin admin] Validating token access rules against request {{(pid=91941) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:04:44.968695 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67595b39-e11b-4e01-9a05-f539aca97ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b433ad0233c4e5dbe8fad9896962e25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53048925682f4806b3831f88058d3439', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91941) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:04:44.969561 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67595b39-e11b-4e01-9a05-f539aca97ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:04:44.970082 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67595b39-e11b-4e01-9a05-f539aca97ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:04:44.970221 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67595b39-e11b-4e01-9a05-f539aca97ae4 admin admin] PATH_INFO: `/v3/projects/c4f78273b11a4d5c84d1cf4ad58aa99c` {{(pid=91941) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:04:44.971348 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67595b39-e11b-4e01-9a05-f539aca97ae4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4f78273b11a4d5c84d1cf4ad58aa99c)` {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:04:44.974679 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67595b39-e11b-4e01-9a05-f539aca97ae4 admin admin] RBAC: Authorization granted {{(pid=91941) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:04:44.984574 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:04:45.026532 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-67595b39-e11b-4e01-9a05-f539aca97ae4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4f78273b11a4d5c84d1cf4ad58aa99c', 'request_id': 'req-67595b39-e11b-4e01-9a05-f539aca97ae4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:45.032793 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-67595b39-e11b-4e01-9a05-f539aca97ae4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4f78273b11a4d5c84d1cf4ad58aa99c', 'request_id': 'req-67595b39-e11b-4e01-9a05-f539aca97ae4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:45.038410 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-67595b39-e11b-4e01-9a05-f539aca97ae4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4f78273b11a4d5c84d1cf4ad58aa99c', 'request_id': 'req-67595b39-e11b-4e01-9a05-f539aca97ae4'} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:45.055796 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-67595b39-e11b-4e01-9a05-f539aca97ae4 admin admin] The token cache is being invalidate because project c4f78273b11a4d5c84d1cf4ad58aa99c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91941) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 16:04:45.056095 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-67595b39-e11b-4e01-9a05-f539aca97ae4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:45.056776 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG keystone.notifications [None req-67595b39-e11b-4e01-9a05-f539aca97ae4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91941) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 16:04:45.058405 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: [pid: 91941|app: 0|req: 1439/2907] 10.222.0.20 () {58 vars in 1232 bytes} [Mon Jun 27 16:04:44 2022] DELETE /identity/v3/projects/c4f78273b11a4d5c84d1cf4ad58aa99c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 16:04:54.708050 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:04:55.051961 dv-cinder-847801-1-iscsi devstack@keystone.service[91941]: DEBUG dbcounter [-] [91941] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91941) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 16:07:09.593749 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e89a6-c396-4cc1-9154-87b999993273 admin admin] Authenticating user token {{(pid=91942) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 16:07:09.595136 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-8f8e89a6-c396-4cc1-9154-87b999993273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:07:09.668300 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e89a6-c396-4cc1-9154-87b999993273 admin admin] Validating token access rules against request {{(pid=91942) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 16:07:09.729323 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f797ab1-98ef-4334-838b-f71969733fe0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffafb68f1bb941588a5217ad83ad907c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7be83fb6f20341139147373837320148', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91942) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 16:07:09.730005 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f797ab1-98ef-4334-838b-f71969733fe0 service placement] REQUEST_METHOD: `GET` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 16:07:09.730291 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f797ab1-98ef-4334-838b-f71969733fe0 service placement] SCRIPT_NAME: `/identity` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 16:07:09.730572 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f797ab1-98ef-4334-838b-f71969733fe0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91942) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 16:07:09.732672 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.fernet_utils [None req-6f797ab1-98ef-4334-838b-f71969733fe0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91942) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 16:07:09.745246 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f797ab1-98ef-4334-838b-f71969733fe0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 16:07:09.746869 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f797ab1-98ef-4334-838b-f71969733fe0 service placement] RBAC: Authorization granted {{(pid=91942) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 16:07:09.756802 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: [pid: 91942|app: 0|req: 1469/2908] 10.222.0.20 () {60 vars in 1365 bytes} [Mon Jun 27 16:07:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 16:07:19.752507 dv-cinder-847801-1-iscsi devstack@keystone.service[91942]: DEBUG dbcounter [-] [91942] Writing DB stats keystone:SELECT=58 {{(pid=91942) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}